You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Bill Graham (Created) (JIRA)" <ji...@apache.org> on 2012/02/22 00:02:49 UTC
[jira] [Created] (PIG-2547) Easier UDFs: Convenient EvalFunc
super-classes
Easier UDFs: Convenient EvalFunc super-classes
----------------------------------------------
Key: PIG-2547
URL: https://issues.apache.org/jira/browse/PIG-2547
Project: Pig
Issue Type: New Feature
Reporter: Bill Graham
Assignee: Bill Graham
We've got a few abstract extensions of EvalFunc that make life easier. If people are interested we can push said classes into Pig.
There are classes, each extending the next. Class naming is all TBD.
* {{TypedOutputEvalFunc<OUT>}} - Implements {{public Schema outputSchema(Schema input)}} based on the generic type of the subclass. Provides common helper validation functions which increment counters for good and bad Tuple data passed. Useful where the input to be worked on is a tuple of size N or greater.
* {{PrimitiveEvalFunc<IN, OUT>}} - Same as above with helper validation allowing the ability it subclass and just implement {{public OUT exec(IN input)}}, where IN and OUT are primitives. Useful when the input is a single primitive in position 0 of a tuple.
* {{FunctionWrapperEvalFunc}} - Wraps a Guava Function implementation (http://guava-libraries.googlecode.com/svn/trunk/javadoc/com/google/common/base/Function.html) and allows UDFs to be used in Pig scripts like so, where {{MyFunction}} is a class that implements {{Function}}:
{noformat}
DEFINE myUdf org.apache.pig.FunctionWrapperEvalFunc('MyFunction')
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2547) Easier UDFs: Convenient EvalFunc
super-classes
Posted by "Dmitriy V. Ryaboy (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257649#comment-13257649 ]
Dmitriy V. Ryaboy commented on PIG-2547:
----------------------------------------
Bumping for review
> Easier UDFs: Convenient EvalFunc super-classes
> ----------------------------------------------
>
> Key: PIG-2547
> URL: https://issues.apache.org/jira/browse/PIG-2547
> Project: Pig
> Issue Type: New Feature
> Reporter: Bill Graham
> Assignee: Bill Graham
> Labels: 0.10_blocker
> Fix For: 0.10.0, 0.11
>
> Attachments: PIG_2547.1.patch, PIG_2547.2.patch
>
>
> We've got a few abstract extensions of EvalFunc that make life easier. If people are interested we can push said classes into Pig.
> There are 3 classes, each extending the next. Class naming is all TBD.
> * {{TypedOutputEvalFunc<OUT>}} - Implements {{public Schema outputSchema(Schema input)}} based on the generic type of the subclass. Provides common helper validation functions which increment counters for good and bad Tuple data passed. Useful where the input to be worked on is a tuple of size N or greater.
> * {{PrimitiveEvalFunc<IN, OUT>}} - Same as above with helper validation allowing the ability it subclass and just implement {{public OUT exec(IN input)}}, where IN and OUT are primitives. Useful when the input is a single primitive in position 0 of a tuple.
> * {{FunctionWrapperEvalFunc}} - Wraps a Guava Function implementation (http://guava-libraries.googlecode.com/svn/trunk/javadoc/com/google/common/base/Function.html) and allows UDFs to be used in Pig scripts like so, where {{MyFunction}} is a class that implements {{Function}}:
> {noformat}
> DEFINE myUdf org.apache.pig.FunctionWrapperEvalFunc('MyFunction')
> {noformat}
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2547) Easier UDFs: Convenient EvalFunc
super-classes
Posted by "Jonathan Coveney (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Coveney updated PIG-2547:
----------------------------------
Fix Version/s: 0.11
0.10
> Easier UDFs: Convenient EvalFunc super-classes
> ----------------------------------------------
>
> Key: PIG-2547
> URL: https://issues.apache.org/jira/browse/PIG-2547
> Project: Pig
> Issue Type: New Feature
> Reporter: Bill Graham
> Assignee: Bill Graham
> Fix For: 0.10, 0.11
>
> Attachments: PIG_2547.1.patch, PIG_2547.2.patch
>
>
> We've got a few abstract extensions of EvalFunc that make life easier. If people are interested we can push said classes into Pig.
> There are 3 classes, each extending the next. Class naming is all TBD.
> * {{TypedOutputEvalFunc<OUT>}} - Implements {{public Schema outputSchema(Schema input)}} based on the generic type of the subclass. Provides common helper validation functions which increment counters for good and bad Tuple data passed. Useful where the input to be worked on is a tuple of size N or greater.
> * {{PrimitiveEvalFunc<IN, OUT>}} - Same as above with helper validation allowing the ability it subclass and just implement {{public OUT exec(IN input)}}, where IN and OUT are primitives. Useful when the input is a single primitive in position 0 of a tuple.
> * {{FunctionWrapperEvalFunc}} - Wraps a Guava Function implementation (http://guava-libraries.googlecode.com/svn/trunk/javadoc/com/google/common/base/Function.html) and allows UDFs to be used in Pig scripts like so, where {{MyFunction}} is a class that implements {{Function}}:
> {noformat}
> DEFINE myUdf org.apache.pig.FunctionWrapperEvalFunc('MyFunction')
> {noformat}
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2547) Easier UDFs: Convenient EvalFunc
super-classes
Posted by "Bill Graham (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-2547:
-----------------------------
Attachment: PIG_2547.1.patch
Here's a first pass at a patch. It includes a number of new files:
{noformat}
src/org/apache/pig/ExceptionalFunction.java
src/org/apache/pig/Function.java
src/org/apache/pig/FunctionWrapperEvalFunc.java
src/org/apache/pig/PrimitiveEvalFunc.java
src/org/apache/pig/TypedOutputEvalFunc.java
test/org/apache/pig/TestFunctionWrapperEvalFunc.java
test/org/apache/pig/TestPrimitiveEvalFunc.java
test/org/apache/pig/TestTypedOutputEvalFunc.java
{noformat}
The new Pig {{Function}} interface was added as a common subinterface to Googles {{Function}} and a new Pig {{ExceptionalFunction}}.
I went through some pains to support all three interfaces in {{FunctionWrapperEvalFunc}}. (Dmitriy, this is different than our impl, which doesn't include support for Google's {{Function}}.)
Please take a look and let me know what you think.
> Easier UDFs: Convenient EvalFunc super-classes
> ----------------------------------------------
>
> Key: PIG-2547
> URL: https://issues.apache.org/jira/browse/PIG-2547
> Project: Pig
> Issue Type: New Feature
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG_2547.1.patch
>
>
> We've got a few abstract extensions of EvalFunc that make life easier. If people are interested we can push said classes into Pig.
> There are 3 classes, each extending the next. Class naming is all TBD.
> * {{TypedOutputEvalFunc<OUT>}} - Implements {{public Schema outputSchema(Schema input)}} based on the generic type of the subclass. Provides common helper validation functions which increment counters for good and bad Tuple data passed. Useful where the input to be worked on is a tuple of size N or greater.
> * {{PrimitiveEvalFunc<IN, OUT>}} - Same as above with helper validation allowing the ability it subclass and just implement {{public OUT exec(IN input)}}, where IN and OUT are primitives. Useful when the input is a single primitive in position 0 of a tuple.
> * {{FunctionWrapperEvalFunc}} - Wraps a Guava Function implementation (http://guava-libraries.googlecode.com/svn/trunk/javadoc/com/google/common/base/Function.html) and allows UDFs to be used in Pig scripts like so, where {{MyFunction}} is a class that implements {{Function}}:
> {noformat}
> DEFINE myUdf org.apache.pig.FunctionWrapperEvalFunc('MyFunction')
> {noformat}
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2547) Easier UDFs: Convenient EvalFunc
super-classes
Posted by "Dmitriy V. Ryaboy (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13213146#comment-13213146 ]
Dmitriy V. Ryaboy commented on PIG-2547:
----------------------------------------
This is kind of a group contrib -- Jimmy Lin, Self, Bill Graham, possibly even some Kevin Weil code in there.
Can non-Twitter check this out and comment? We've found this structure quite useful for making our UDFs light-weight. Abstracting common processing functionality into guava Functions lets us share business logic between multiple systems, instead of embedding it inside UDFs. We think it's pretty neat :).
> Easier UDFs: Convenient EvalFunc super-classes
> ----------------------------------------------
>
> Key: PIG-2547
> URL: https://issues.apache.org/jira/browse/PIG-2547
> Project: Pig
> Issue Type: New Feature
> Reporter: Bill Graham
> Assignee: Bill Graham
>
> We've got a few abstract extensions of EvalFunc that make life easier. If people are interested we can push said classes into Pig.
> There are 3 classes, each extending the next. Class naming is all TBD.
> * {{TypedOutputEvalFunc<OUT>}} - Implements {{public Schema outputSchema(Schema input)}} based on the generic type of the subclass. Provides common helper validation functions which increment counters for good and bad Tuple data passed. Useful where the input to be worked on is a tuple of size N or greater.
> * {{PrimitiveEvalFunc<IN, OUT>}} - Same as above with helper validation allowing the ability it subclass and just implement {{public OUT exec(IN input)}}, where IN and OUT are primitives. Useful when the input is a single primitive in position 0 of a tuple.
> * {{FunctionWrapperEvalFunc}} - Wraps a Guava Function implementation (http://guava-libraries.googlecode.com/svn/trunk/javadoc/com/google/common/base/Function.html) and allows UDFs to be used in Pig scripts like so, where {{MyFunction}} is a class that implements {{Function}}:
> {noformat}
> DEFINE myUdf org.apache.pig.FunctionWrapperEvalFunc('MyFunction')
> {noformat}
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2547) Easier UDFs: Convenient EvalFunc
super-classes
Posted by "Bill Graham (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-2547:
-----------------------------
Status: Patch Available (was: Open)
> Easier UDFs: Convenient EvalFunc super-classes
> ----------------------------------------------
>
> Key: PIG-2547
> URL: https://issues.apache.org/jira/browse/PIG-2547
> Project: Pig
> Issue Type: New Feature
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG_2547.1.patch
>
>
> We've got a few abstract extensions of EvalFunc that make life easier. If people are interested we can push said classes into Pig.
> There are 3 classes, each extending the next. Class naming is all TBD.
> * {{TypedOutputEvalFunc<OUT>}} - Implements {{public Schema outputSchema(Schema input)}} based on the generic type of the subclass. Provides common helper validation functions which increment counters for good and bad Tuple data passed. Useful where the input to be worked on is a tuple of size N or greater.
> * {{PrimitiveEvalFunc<IN, OUT>}} - Same as above with helper validation allowing the ability it subclass and just implement {{public OUT exec(IN input)}}, where IN and OUT are primitives. Useful when the input is a single primitive in position 0 of a tuple.
> * {{FunctionWrapperEvalFunc}} - Wraps a Guava Function implementation (http://guava-libraries.googlecode.com/svn/trunk/javadoc/com/google/common/base/Function.html) and allows UDFs to be used in Pig scripts like so, where {{MyFunction}} is a class that implements {{Function}}:
> {noformat}
> DEFINE myUdf org.apache.pig.FunctionWrapperEvalFunc('MyFunction')
> {noformat}
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2547) Easier UDFs: Convenient EvalFunc
super-classes
Posted by "Bill Graham (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13213163#comment-13213163 ]
Bill Graham commented on PIG-2547:
----------------------------------
No uploaded patch yet to look at, still in the process of refactoring.
> Easier UDFs: Convenient EvalFunc super-classes
> ----------------------------------------------
>
> Key: PIG-2547
> URL: https://issues.apache.org/jira/browse/PIG-2547
> Project: Pig
> Issue Type: New Feature
> Reporter: Bill Graham
> Assignee: Bill Graham
>
> We've got a few abstract extensions of EvalFunc that make life easier. If people are interested we can push said classes into Pig.
> There are 3 classes, each extending the next. Class naming is all TBD.
> * {{TypedOutputEvalFunc<OUT>}} - Implements {{public Schema outputSchema(Schema input)}} based on the generic type of the subclass. Provides common helper validation functions which increment counters for good and bad Tuple data passed. Useful where the input to be worked on is a tuple of size N or greater.
> * {{PrimitiveEvalFunc<IN, OUT>}} - Same as above with helper validation allowing the ability it subclass and just implement {{public OUT exec(IN input)}}, where IN and OUT are primitives. Useful when the input is a single primitive in position 0 of a tuple.
> * {{FunctionWrapperEvalFunc}} - Wraps a Guava Function implementation (http://guava-libraries.googlecode.com/svn/trunk/javadoc/com/google/common/base/Function.html) and allows UDFs to be used in Pig scripts like so, where {{MyFunction}} is a class that implements {{Function}}:
> {noformat}
> DEFINE myUdf org.apache.pig.FunctionWrapperEvalFunc('MyFunction')
> {noformat}
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2547) Easier UDFs: Convenient EvalFunc
super-classes
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273673#comment-13273673 ]
Daniel Dai commented on PIG-2547:
---------------------------------
+1, please commit to trunk.
> Easier UDFs: Convenient EvalFunc super-classes
> ----------------------------------------------
>
> Key: PIG-2547
> URL: https://issues.apache.org/jira/browse/PIG-2547
> Project: Pig
> Issue Type: New Feature
> Reporter: Bill Graham
> Assignee: Bill Graham
> Fix For: 0.11
>
> Attachments: PIG-2547.3.patch, PIG_2547.1.patch, PIG_2547.2.patch
>
>
> We've got a few abstract extensions of EvalFunc that make life easier. If people are interested we can push said classes into Pig.
> There are 3 classes, each extending the next. Class naming is all TBD.
> * {{TypedOutputEvalFunc<OUT>}} - Implements {{public Schema outputSchema(Schema input)}} based on the generic type of the subclass. Provides common helper validation functions which increment counters for good and bad Tuple data passed. Useful where the input to be worked on is a tuple of size N or greater.
> * {{PrimitiveEvalFunc<IN, OUT>}} - Same as above with helper validation allowing the ability it subclass and just implement {{public OUT exec(IN input)}}, where IN and OUT are primitives. Useful when the input is a single primitive in position 0 of a tuple.
> * {{FunctionWrapperEvalFunc}} - Wraps a Guava Function implementation (http://guava-libraries.googlecode.com/svn/trunk/javadoc/com/google/common/base/Function.html) and allows UDFs to be used in Pig scripts like so, where {{MyFunction}} is a class that implements {{Function}}:
> {noformat}
> DEFINE myUdf org.apache.pig.FunctionWrapperEvalFunc('MyFunction')
> {noformat}
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2547) Easier UDFs: Convenient EvalFunc
super-classes
Posted by "Bill Graham (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-2547:
-----------------------------
Description:
We've got a few abstract extensions of EvalFunc that make life easier. If people are interested we can push said classes into Pig.
There are 3 classes, each extending the next. Class naming is all TBD.
* {{TypedOutputEvalFunc<OUT>}} - Implements {{public Schema outputSchema(Schema input)}} based on the generic type of the subclass. Provides common helper validation functions which increment counters for good and bad Tuple data passed. Useful where the input to be worked on is a tuple of size N or greater.
* {{PrimitiveEvalFunc<IN, OUT>}} - Same as above with helper validation allowing the ability it subclass and just implement {{public OUT exec(IN input)}}, where IN and OUT are primitives. Useful when the input is a single primitive in position 0 of a tuple.
* {{FunctionWrapperEvalFunc}} - Wraps a Guava Function implementation (http://guava-libraries.googlecode.com/svn/trunk/javadoc/com/google/common/base/Function.html) and allows UDFs to be used in Pig scripts like so, where {{MyFunction}} is a class that implements {{Function}}:
{noformat}
DEFINE myUdf org.apache.pig.FunctionWrapperEvalFunc('MyFunction')
{noformat}
was:
We've got a few abstract extensions of EvalFunc that make life easier. If people are interested we can push said classes into Pig.
There are classes, each extending the next. Class naming is all TBD.
* {{TypedOutputEvalFunc<OUT>}} - Implements {{public Schema outputSchema(Schema input)}} based on the generic type of the subclass. Provides common helper validation functions which increment counters for good and bad Tuple data passed. Useful where the input to be worked on is a tuple of size N or greater.
* {{PrimitiveEvalFunc<IN, OUT>}} - Same as above with helper validation allowing the ability it subclass and just implement {{public OUT exec(IN input)}}, where IN and OUT are primitives. Useful when the input is a single primitive in position 0 of a tuple.
* {{FunctionWrapperEvalFunc}} - Wraps a Guava Function implementation (http://guava-libraries.googlecode.com/svn/trunk/javadoc/com/google/common/base/Function.html) and allows UDFs to be used in Pig scripts like so, where {{MyFunction}} is a class that implements {{Function}}:
{noformat}
DEFINE myUdf org.apache.pig.FunctionWrapperEvalFunc('MyFunction')
{noformat}
> Easier UDFs: Convenient EvalFunc super-classes
> ----------------------------------------------
>
> Key: PIG-2547
> URL: https://issues.apache.org/jira/browse/PIG-2547
> Project: Pig
> Issue Type: New Feature
> Reporter: Bill Graham
> Assignee: Bill Graham
>
> We've got a few abstract extensions of EvalFunc that make life easier. If people are interested we can push said classes into Pig.
> There are 3 classes, each extending the next. Class naming is all TBD.
> * {{TypedOutputEvalFunc<OUT>}} - Implements {{public Schema outputSchema(Schema input)}} based on the generic type of the subclass. Provides common helper validation functions which increment counters for good and bad Tuple data passed. Useful where the input to be worked on is a tuple of size N or greater.
> * {{PrimitiveEvalFunc<IN, OUT>}} - Same as above with helper validation allowing the ability it subclass and just implement {{public OUT exec(IN input)}}, where IN and OUT are primitives. Useful when the input is a single primitive in position 0 of a tuple.
> * {{FunctionWrapperEvalFunc}} - Wraps a Guava Function implementation (http://guava-libraries.googlecode.com/svn/trunk/javadoc/com/google/common/base/Function.html) and allows UDFs to be used in Pig scripts like so, where {{MyFunction}} is a class that implements {{Function}}:
> {noformat}
> DEFINE myUdf org.apache.pig.FunctionWrapperEvalFunc('MyFunction')
> {noformat}
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2547) Easier UDFs: Convenient EvalFunc
super-classes
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-2547:
-----------------------------
Fix Version/s: (was: 0.10.1)
> Easier UDFs: Convenient EvalFunc super-classes
> ----------------------------------------------
>
> Key: PIG-2547
> URL: https://issues.apache.org/jira/browse/PIG-2547
> Project: Pig
> Issue Type: New Feature
> Reporter: Bill Graham
> Assignee: Bill Graham
> Fix For: 0.11
>
> Attachments: PIG-2547.3.patch, PIG_2547.1.patch, PIG_2547.2.patch
>
>
> We've got a few abstract extensions of EvalFunc that make life easier. If people are interested we can push said classes into Pig.
> There are 3 classes, each extending the next. Class naming is all TBD.
> * {{TypedOutputEvalFunc<OUT>}} - Implements {{public Schema outputSchema(Schema input)}} based on the generic type of the subclass. Provides common helper validation functions which increment counters for good and bad Tuple data passed. Useful where the input to be worked on is a tuple of size N or greater.
> * {{PrimitiveEvalFunc<IN, OUT>}} - Same as above with helper validation allowing the ability it subclass and just implement {{public OUT exec(IN input)}}, where IN and OUT are primitives. Useful when the input is a single primitive in position 0 of a tuple.
> * {{FunctionWrapperEvalFunc}} - Wraps a Guava Function implementation (http://guava-libraries.googlecode.com/svn/trunk/javadoc/com/google/common/base/Function.html) and allows UDFs to be used in Pig scripts like so, where {{MyFunction}} is a class that implements {{Function}}:
> {noformat}
> DEFINE myUdf org.apache.pig.FunctionWrapperEvalFunc('MyFunction')
> {noformat}
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2547) Easier UDFs: Convenient EvalFunc
super-classes
Posted by "Bill Graham (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-2547:
-----------------------------
Attachment: PIG_2547.2.patch
Attaching a 2nd patch, please disregard the 1st.
> Easier UDFs: Convenient EvalFunc super-classes
> ----------------------------------------------
>
> Key: PIG-2547
> URL: https://issues.apache.org/jira/browse/PIG-2547
> Project: Pig
> Issue Type: New Feature
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG_2547.1.patch, PIG_2547.2.patch
>
>
> We've got a few abstract extensions of EvalFunc that make life easier. If people are interested we can push said classes into Pig.
> There are 3 classes, each extending the next. Class naming is all TBD.
> * {{TypedOutputEvalFunc<OUT>}} - Implements {{public Schema outputSchema(Schema input)}} based on the generic type of the subclass. Provides common helper validation functions which increment counters for good and bad Tuple data passed. Useful where the input to be worked on is a tuple of size N or greater.
> * {{PrimitiveEvalFunc<IN, OUT>}} - Same as above with helper validation allowing the ability it subclass and just implement {{public OUT exec(IN input)}}, where IN and OUT are primitives. Useful when the input is a single primitive in position 0 of a tuple.
> * {{FunctionWrapperEvalFunc}} - Wraps a Guava Function implementation (http://guava-libraries.googlecode.com/svn/trunk/javadoc/com/google/common/base/Function.html) and allows UDFs to be used in Pig scripts like so, where {{MyFunction}} is a class that implements {{Function}}:
> {noformat}
> DEFINE myUdf org.apache.pig.FunctionWrapperEvalFunc('MyFunction')
> {noformat}
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2547) Easier UDFs: Convenient EvalFunc
super-classes
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-2547:
----------------------------
Fix Version/s: (was: 0.10.0)
0.10.1
> Easier UDFs: Convenient EvalFunc super-classes
> ----------------------------------------------
>
> Key: PIG-2547
> URL: https://issues.apache.org/jira/browse/PIG-2547
> Project: Pig
> Issue Type: New Feature
> Reporter: Bill Graham
> Assignee: Bill Graham
> Labels: 0.10_blocker
> Fix For: 0.11, 0.10.1
>
> Attachments: PIG_2547.1.patch, PIG_2547.2.patch
>
>
> We've got a few abstract extensions of EvalFunc that make life easier. If people are interested we can push said classes into Pig.
> There are 3 classes, each extending the next. Class naming is all TBD.
> * {{TypedOutputEvalFunc<OUT>}} - Implements {{public Schema outputSchema(Schema input)}} based on the generic type of the subclass. Provides common helper validation functions which increment counters for good and bad Tuple data passed. Useful where the input to be worked on is a tuple of size N or greater.
> * {{PrimitiveEvalFunc<IN, OUT>}} - Same as above with helper validation allowing the ability it subclass and just implement {{public OUT exec(IN input)}}, where IN and OUT are primitives. Useful when the input is a single primitive in position 0 of a tuple.
> * {{FunctionWrapperEvalFunc}} - Wraps a Guava Function implementation (http://guava-libraries.googlecode.com/svn/trunk/javadoc/com/google/common/base/Function.html) and allows UDFs to be used in Pig scripts like so, where {{MyFunction}} is a class that implements {{Function}}:
> {noformat}
> DEFINE myUdf org.apache.pig.FunctionWrapperEvalFunc('MyFunction')
> {noformat}
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2547) Easier UDFs: Convenient EvalFunc
super-classes
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-2547:
-----------------------------
Attachment: PIG-2547.3.patch
Thanks Daniel, here's a 3rd parch with your comments incorporated. Note that there is a Guava Function unit test (see IntegerFloatFunction).
> Easier UDFs: Convenient EvalFunc super-classes
> ----------------------------------------------
>
> Key: PIG-2547
> URL: https://issues.apache.org/jira/browse/PIG-2547
> Project: Pig
> Issue Type: New Feature
> Reporter: Bill Graham
> Assignee: Bill Graham
> Fix For: 0.11, 0.10.1
>
> Attachments: PIG-2547.3.patch, PIG_2547.1.patch, PIG_2547.2.patch
>
>
> We've got a few abstract extensions of EvalFunc that make life easier. If people are interested we can push said classes into Pig.
> There are 3 classes, each extending the next. Class naming is all TBD.
> * {{TypedOutputEvalFunc<OUT>}} - Implements {{public Schema outputSchema(Schema input)}} based on the generic type of the subclass. Provides common helper validation functions which increment counters for good and bad Tuple data passed. Useful where the input to be worked on is a tuple of size N or greater.
> * {{PrimitiveEvalFunc<IN, OUT>}} - Same as above with helper validation allowing the ability it subclass and just implement {{public OUT exec(IN input)}}, where IN and OUT are primitives. Useful when the input is a single primitive in position 0 of a tuple.
> * {{FunctionWrapperEvalFunc}} - Wraps a Guava Function implementation (http://guava-libraries.googlecode.com/svn/trunk/javadoc/com/google/common/base/Function.html) and allows UDFs to be used in Pig scripts like so, where {{MyFunction}} is a class that implements {{Function}}:
> {noformat}
> DEFINE myUdf org.apache.pig.FunctionWrapperEvalFunc('MyFunction')
> {noformat}
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2547) Easier UDFs: Convenient EvalFunc
super-classes
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13260338#comment-13260338 ]
Daniel Dai commented on PIG-2547:
---------------------------------
Looks good, couple of comments:
1. TypedOutputEvalFunc: Get output schema from generic type is already part of EvalFunc, besides, EvalFunc.outputSchema can also handle annotation now, do we still need TypedOutputEvalFunc.outputSchema?
2. FunctionWrapperEvalFunc: better to be in builtin package, so that user can omit package name in Pig script. Also, there seems no coverage for wrapping guava function in TestFunctionWrapperEvalFunc, better to have some.
> Easier UDFs: Convenient EvalFunc super-classes
> ----------------------------------------------
>
> Key: PIG-2547
> URL: https://issues.apache.org/jira/browse/PIG-2547
> Project: Pig
> Issue Type: New Feature
> Reporter: Bill Graham
> Assignee: Bill Graham
> Labels: 0.10_blocker
> Fix For: 0.10.0, 0.11
>
> Attachments: PIG_2547.1.patch, PIG_2547.2.patch
>
>
> We've got a few abstract extensions of EvalFunc that make life easier. If people are interested we can push said classes into Pig.
> There are 3 classes, each extending the next. Class naming is all TBD.
> * {{TypedOutputEvalFunc<OUT>}} - Implements {{public Schema outputSchema(Schema input)}} based on the generic type of the subclass. Provides common helper validation functions which increment counters for good and bad Tuple data passed. Useful where the input to be worked on is a tuple of size N or greater.
> * {{PrimitiveEvalFunc<IN, OUT>}} - Same as above with helper validation allowing the ability it subclass and just implement {{public OUT exec(IN input)}}, where IN and OUT are primitives. Useful when the input is a single primitive in position 0 of a tuple.
> * {{FunctionWrapperEvalFunc}} - Wraps a Guava Function implementation (http://guava-libraries.googlecode.com/svn/trunk/javadoc/com/google/common/base/Function.html) and allows UDFs to be used in Pig scripts like so, where {{MyFunction}} is a class that implements {{Function}}:
> {noformat}
> DEFINE myUdf org.apache.pig.FunctionWrapperEvalFunc('MyFunction')
> {noformat}
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2547) Easier UDFs: Convenient EvalFunc
super-classes
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-2547:
-----------------------------
Resolution: Fixed
Release Note: New superclasses to make it easier to implement UDFs.
Status: Resolved (was: Patch Available)
Committed
> Easier UDFs: Convenient EvalFunc super-classes
> ----------------------------------------------
>
> Key: PIG-2547
> URL: https://issues.apache.org/jira/browse/PIG-2547
> Project: Pig
> Issue Type: New Feature
> Reporter: Bill Graham
> Assignee: Bill Graham
> Fix For: 0.11
>
> Attachments: PIG-2547.3.patch, PIG_2547.1.patch, PIG_2547.2.patch
>
>
> We've got a few abstract extensions of EvalFunc that make life easier. If people are interested we can push said classes into Pig.
> There are 3 classes, each extending the next. Class naming is all TBD.
> * {{TypedOutputEvalFunc<OUT>}} - Implements {{public Schema outputSchema(Schema input)}} based on the generic type of the subclass. Provides common helper validation functions which increment counters for good and bad Tuple data passed. Useful where the input to be worked on is a tuple of size N or greater.
> * {{PrimitiveEvalFunc<IN, OUT>}} - Same as above with helper validation allowing the ability it subclass and just implement {{public OUT exec(IN input)}}, where IN and OUT are primitives. Useful when the input is a single primitive in position 0 of a tuple.
> * {{FunctionWrapperEvalFunc}} - Wraps a Guava Function implementation (http://guava-libraries.googlecode.com/svn/trunk/javadoc/com/google/common/base/Function.html) and allows UDFs to be used in Pig scripts like so, where {{MyFunction}} is a class that implements {{Function}}:
> {noformat}
> DEFINE myUdf org.apache.pig.FunctionWrapperEvalFunc('MyFunction')
> {noformat}
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira