You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Dmitriy V. Ryaboy (Created) (JIRA)" <ji...@apache.org> on 2011/12/07 07:16:39 UTC
[jira] [Created] (PIG-2403) Reduce code duplication in SUM, MAX,
MIN udfs
Reduce code duplication in SUM, MAX, MIN udfs
---------------------------------------------
Key: PIG-2403
URL: https://issues.apache.org/jira/browse/PIG-2403
Project: Pig
Issue Type: Improvement
Reporter: Dmitriy V. Ryaboy
Assignee: Dmitriy V. Ryaboy
Fix For: 0.11
The current type-specific implementations of Max, Min, and Sum have a lot of duplicated code. We can reduce this significantly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2403) Reduce code duplication in SUM, MAX,
MIN udfs
Posted by "Dmitriy V. Ryaboy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-2403:
-----------------------------------
Status: Patch Available (was: Open)
> Reduce code duplication in SUM, MAX, MIN udfs
> ---------------------------------------------
>
> Key: PIG-2403
> URL: https://issues.apache.org/jira/browse/PIG-2403
> Project: Pig
> Issue Type: Improvement
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Fix For: 0.11
>
> Attachments: PIG-2403.patch
>
>
> The current type-specific implementations of Max, Min, and Sum have a lot of duplicated code. We can reduce this significantly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2403) Reduce code duplication in SUM, MAX,
MIN udfs
Posted by "Dmitriy V. Ryaboy (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167976#comment-13167976 ]
Dmitriy V. Ryaboy commented on PIG-2403:
----------------------------------------
Please review :)
> Reduce code duplication in SUM, MAX, MIN udfs
> ---------------------------------------------
>
> Key: PIG-2403
> URL: https://issues.apache.org/jira/browse/PIG-2403
> Project: Pig
> Issue Type: Improvement
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Fix For: 0.11
>
> Attachments: PIG-2403.patch
>
>
> The current type-specific implementations of Max, Min, and Sum have a lot of duplicated code. We can reduce this significantly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2403) Reduce code duplication in SUM, MAX,
MIN udfs
Posted by "Dmitriy V. Ryaboy (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164184#comment-13164184 ]
Dmitriy V. Ryaboy commented on PIG-2403:
----------------------------------------
Summary of changes:
$ git diff apache/trunk --stat
.../pig/builtin/AlgebraicByteArrayMathBase.java | 213 +++++++++++++++++++
.../pig/builtin/AlgebraicDoubleMathBase.java | 166 +++++++++++++++
.../apache/pig/builtin/AlgebraicFloatMathBase.java | 165 +++++++++++++++
.../apache/pig/builtin/AlgebraicIntMathBase.java | 167 +++++++++++++++
.../apache/pig/builtin/AlgebraicLongMathBase.java | 168 +++++++++++++++
src/org/apache/pig/builtin/AlgebraicMathBase.java | 127 ++++++++++++
src/org/apache/pig/builtin/DoubleAbs.java | 35 +---
src/org/apache/pig/builtin/DoubleMax.java | 169 +---------------
src/org/apache/pig/builtin/DoubleMin.java | 165 +--------------
src/org/apache/pig/builtin/DoubleSum.java | 168 +--------------
src/org/apache/pig/builtin/FloatMax.java | 166 +--------------
src/org/apache/pig/builtin/FloatMin.java | 165 +--------------
src/org/apache/pig/builtin/FloatSum.java | 210 +-------------------
src/org/apache/pig/builtin/IntMax.java | 167 +--------------
src/org/apache/pig/builtin/IntMin.java | 167 +--------------
src/org/apache/pig/builtin/IntSum.java | 210 +-------------------
src/org/apache/pig/builtin/LongMax.java | 166 +--------------
src/org/apache/pig/builtin/LongMin.java | 165 +--------------
src/org/apache/pig/builtin/LongSum.java | 164 +--------------
src/org/apache/pig/builtin/MAX.java | 206 +------------------
src/org/apache/pig/builtin/MIN.java | 207 +------------------
src/org/apache/pig/builtin/SUM.java | 216 ++------------------
22 files changed, 1138 insertions(+), 2614 deletions(-)
> Reduce code duplication in SUM, MAX, MIN udfs
> ---------------------------------------------
>
> Key: PIG-2403
> URL: https://issues.apache.org/jira/browse/PIG-2403
> Project: Pig
> Issue Type: Improvement
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Fix For: 0.11
>
> Attachments: PIG-2403.patch
>
>
> The current type-specific implementations of Max, Min, and Sum have a lot of duplicated code. We can reduce this significantly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2403) Reduce code duplication in SUM, MAX,
MIN udfs
Posted by "Dmitriy V. Ryaboy (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168974#comment-13168974 ]
Dmitriy V. Ryaboy commented on PIG-2403:
----------------------------------------
Works on my mac.. so I don't know what's going on there.
I don't really remember what's going on with Intermediate and Final.. I had to wrestle with java's inheritance quite a bit to get around static methods / classes, I think this one was something I ran out of steam trying to bend around.
I'll commit as is, and save further optimization for future generations.
> Reduce code duplication in SUM, MAX, MIN udfs
> ---------------------------------------------
>
> Key: PIG-2403
> URL: https://issues.apache.org/jira/browse/PIG-2403
> Project: Pig
> Issue Type: Improvement
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Fix For: 0.11
>
> Attachments: PIG-2403.patch
>
>
> The current type-specific implementations of Max, Min, and Sum have a lot of duplicated code. We can reduce this significantly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2403) Reduce code duplication in SUM, MAX,
MIN udfs
Posted by "Alan Gates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168683#comment-13168683 ]
Alan Gates commented on PIG-2403:
---------------------------------
I tried to review the patch, but I can't get it to apply. It says:
{code}
gates> patch -p0 < PIG-2403.patch
patching file src/org/apache/pig/builtin/AlgebraicByteArrayMathBase.java
patching file src/org/apache/pig/builtin/AlgebraicDoubleMathBase.java
patching file src/org/apache/pig/builtin/AlgebraicFloatMathBase.java
patching file src/org/apache/pig/builtin/AlgebraicIntMathBase.java
patching file src/org/apache/pig/builtin/AlgebraicLongMathBase.java
patching file src/org/apache/pig/builtin/AlgebraicMathBase.java
can't find file to patch at input line 1047
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|diff --git src/org/apache/pig/builtin/DoubleAbs.java src/org/apache/pig/builtin/DoubleAbs.java
|index bac4861..79c6be8 100644
|--- src/org/apache/pig/builtin/DoubleAbs.java
|+++ src/org/apache/pig/builtin/DoubleAbs.java
--------------------------
File to patch:
{code}
It looks like maybe you used git diff without the --no-prefix option.
> Reduce code duplication in SUM, MAX, MIN udfs
> ---------------------------------------------
>
> Key: PIG-2403
> URL: https://issues.apache.org/jira/browse/PIG-2403
> Project: Pig
> Issue Type: Improvement
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Fix For: 0.11
>
> Attachments: PIG-2403.patch
>
>
> The current type-specific implementations of Max, Min, and Sum have a lot of duplicated code. We can reduce this significantly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2403) Reduce code duplication in SUM, MAX,
MIN udfs
Posted by "Dmitriy V. Ryaboy (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168689#comment-13168689 ]
Dmitriy V. Ryaboy commented on PIG-2403:
----------------------------------------
I used the no-prefix option (otherwise it wouldn't be able to patch all those other files).
Patch applies fine for my svn trunk checkout, too.. you are sure you are using a clean environment?
{code}
[tw-mbp13-dryaboy pig-svn]$ svn up
At revision 1213916.
[tw-mbp13-dryaboy pig-svn]$ patch -p0 < ../pig-git/patches/PIG-2403.patch
patching file src/org/apache/pig/builtin/AlgebraicByteArrayMathBase.java
patching file src/org/apache/pig/builtin/AlgebraicDoubleMathBase.java
patching file src/org/apache/pig/builtin/AlgebraicFloatMathBase.java
patching file src/org/apache/pig/builtin/AlgebraicIntMathBase.java
patching file src/org/apache/pig/builtin/AlgebraicLongMathBase.java
patching file src/org/apache/pig/builtin/AlgebraicMathBase.java
patching file src/org/apache/pig/builtin/DoubleAbs.java
patching file src/org/apache/pig/builtin/DoubleMax.java
patching file src/org/apache/pig/builtin/DoubleMin.java
patching file src/org/apache/pig/builtin/DoubleSum.java
patching file src/org/apache/pig/builtin/FloatMax.java
patching file src/org/apache/pig/builtin/FloatMin.java
patching file src/org/apache/pig/builtin/FloatSum.java
patching file src/org/apache/pig/builtin/IntMax.java
patching file src/org/apache/pig/builtin/IntMin.java
patching file src/org/apache/pig/builtin/IntSum.java
patching file src/org/apache/pig/builtin/LongMax.java
patching file src/org/apache/pig/builtin/LongMin.java
patching file src/org/apache/pig/builtin/LongSum.java
patching file src/org/apache/pig/builtin/MAX.java
patching file src/org/apache/pig/builtin/MIN.java
patching file src/org/apache/pig/builtin/SUM.java
{code}
> Reduce code duplication in SUM, MAX, MIN udfs
> ---------------------------------------------
>
> Key: PIG-2403
> URL: https://issues.apache.org/jira/browse/PIG-2403
> Project: Pig
> Issue Type: Improvement
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Fix For: 0.11
>
> Attachments: PIG-2403.patch
>
>
> The current type-specific implementations of Max, Min, and Sum have a lot of duplicated code. We can reduce this significantly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2403) Reduce code duplication in SUM, MAX,
MIN udfs
Posted by "Dmitriy V. Ryaboy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-2403:
-----------------------------------
Attachment: PIG-2403.patch
The attached patch extracts common code from max, sum, and min implementations into a common base class and reuses it across the board. I'm not totally satisfied -- a lot of boilerplate is still copy-pasted between the long/int/float/double implementations, but it's a step in the right direction.
TestBuiltin passes.
Ready for review.
> Reduce code duplication in SUM, MAX, MIN udfs
> ---------------------------------------------
>
> Key: PIG-2403
> URL: https://issues.apache.org/jira/browse/PIG-2403
> Project: Pig
> Issue Type: Improvement
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Fix For: 0.11
>
> Attachments: PIG-2403.patch
>
>
> The current type-specific implementations of Max, Min, and Sum have a lot of duplicated code. We can reduce this significantly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2403) Reduce code duplication in SUM, MAX,
MIN udfs
Posted by "Dmitriy V. Ryaboy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-2403:
-----------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed to trunk.
> Reduce code duplication in SUM, MAX, MIN udfs
> ---------------------------------------------
>
> Key: PIG-2403
> URL: https://issues.apache.org/jira/browse/PIG-2403
> Project: Pig
> Issue Type: Improvement
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Fix For: 0.11
>
> Attachments: PIG-2403.patch
>
>
> The current type-specific implementations of Max, Min, and Sum have a lot of duplicated code. We can reduce this significantly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2403) Reduce code duplication in SUM, MAX,
MIN udfs
Posted by "Alan Gates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168881#comment-13168881 ]
Alan Gates commented on PIG-2403:
---------------------------------
Hmm, it works on my linux box but not my mac. How odd.
I'm not clear on why AlgebraicMathBase had to implement Intermediate and Final. Why not let Java just figure that out via inheritance?
Other than that +1.
All e2e tests that use MIN, MAX, or SUM pass.
> Reduce code duplication in SUM, MAX, MIN udfs
> ---------------------------------------------
>
> Key: PIG-2403
> URL: https://issues.apache.org/jira/browse/PIG-2403
> Project: Pig
> Issue Type: Improvement
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Fix For: 0.11
>
> Attachments: PIG-2403.patch
>
>
> The current type-specific implementations of Max, Min, and Sum have a lot of duplicated code. We can reduce this significantly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira