You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org> on 2011/07/01 05:40:28 UTC
[jira] [Created] (PIG-2151) Add annotation to specify output schema
in Java UDFs
Add annotation to specify output schema in Java UDFs
----------------------------------------------------
Key: PIG-2151
URL: https://issues.apache.org/jira/browse/PIG-2151
Project: Pig
Issue Type: Improvement
Reporter: Dmitriy V. Ryaboy
In Python udfs, the output schema of an eval func is specified via an @OutputSchema annotation.
We should do the same for Java udfs instead of making people write the cumbersome outputSchema functions.
If an annotation exists, it should override an existing outputSchema implementation (since all evalFuncs will inherit the method from EvalFunc.java and there is thus no way to *not* have an implementation).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2151) Add annotation to specify output
schema in Java UDFs
Posted by "Daniel Dai (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149457#comment-13149457 ]
Daniel Dai commented on PIG-2151:
---------------------------------
+1. Don't forget to add release notes.
> Add annotation to specify output schema in Java UDFs
> ----------------------------------------------------
>
> Key: PIG-2151
> URL: https://issues.apache.org/jira/browse/PIG-2151
> Project: Pig
> Issue Type: Improvement
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Attachments: PIG-2151.patch
>
>
> In Python udfs, the output schema of an eval func is specified via an @OutputSchema annotation.
> We should do the same for Java udfs instead of making people write the cumbersome outputSchema functions.
> If an annotation exists, it should override an existing outputSchema implementation (since all evalFuncs will inherit the method from EvalFunc.java and there is thus no way to *not* have an implementation).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2151) Add annotation to specify output
schema in Java UDFs
Posted by "Dmitriy V. Ryaboy (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150261#comment-13150261 ]
Dmitriy V. Ryaboy commented on PIG-2151:
----------------------------------------
Yeah.. file a bug against Utils.getSchemaFromString?
> Add annotation to specify output schema in Java UDFs
> ----------------------------------------------------
>
> Key: PIG-2151
> URL: https://issues.apache.org/jira/browse/PIG-2151
> Project: Pig
> Issue Type: Improvement
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Fix For: 0.11
>
> Attachments: PIG-2151.patch
>
>
> In Python udfs, the output schema of an eval func is specified via an @OutputSchema annotation.
> We should do the same for Java udfs instead of making people write the cumbersome outputSchema functions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2151) Add annotation to specify output
schema in Java UDFs
Posted by "Jonathan Coveney (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150256#comment-13150256 ]
Jonathan Coveney commented on PIG-2151:
---------------------------------------
Something to think about: you can have schema's will null or no name, but in this, doing @OutputSchema("double") will fail. I guess that is a failing of Utils.getSchemaFromString.
> Add annotation to specify output schema in Java UDFs
> ----------------------------------------------------
>
> Key: PIG-2151
> URL: https://issues.apache.org/jira/browse/PIG-2151
> Project: Pig
> Issue Type: Improvement
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Fix For: 0.11
>
> Attachments: PIG-2151.patch
>
>
> In Python udfs, the output schema of an eval func is specified via an @OutputSchema annotation.
> We should do the same for Java udfs instead of making people write the cumbersome outputSchema functions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2151) Add annotation to specify output schema
in Java UDFs
Posted by "Dmitriy V. Ryaboy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-2151:
-----------------------------------
Assignee: Dmitriy V. Ryaboy
Status: Patch Available (was: Open)
> Add annotation to specify output schema in Java UDFs
> ----------------------------------------------------
>
> Key: PIG-2151
> URL: https://issues.apache.org/jira/browse/PIG-2151
> Project: Pig
> Issue Type: Improvement
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Attachments: PIG-2151.patch
>
>
> In Python udfs, the output schema of an eval func is specified via an @OutputSchema annotation.
> We should do the same for Java udfs instead of making people write the cumbersome outputSchema functions.
> If an annotation exists, it should override an existing outputSchema implementation (since all evalFuncs will inherit the method from EvalFunc.java and there is thus no way to *not* have an implementation).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2151) Add annotation to specify output schema
in Java UDFs
Posted by "Dmitriy V. Ryaboy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-2151:
-----------------------------------
Description:
In Python udfs, the output schema of an eval func is specified via an @OutputSchema annotation.
We should do the same for Java udfs instead of making people write the cumbersome outputSchema functions.
was:
In Python udfs, the output schema of an eval func is specified via an @OutputSchema annotation.
We should do the same for Java udfs instead of making people write the cumbersome outputSchema functions.
If an annotation exists, it should override an existing outputSchema implementation (since all evalFuncs will inherit the method from EvalFunc.java and there is thus no way to *not* have an implementation).
> Add annotation to specify output schema in Java UDFs
> ----------------------------------------------------
>
> Key: PIG-2151
> URL: https://issues.apache.org/jira/browse/PIG-2151
> Project: Pig
> Issue Type: Improvement
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Fix For: 0.11
>
> Attachments: PIG-2151.patch
>
>
> In Python udfs, the output schema of an eval func is specified via an @OutputSchema annotation.
> We should do the same for Java udfs instead of making people write the cumbersome outputSchema functions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2151) Add annotation to specify output schema
in Java UDFs
Posted by "Dmitriy V. Ryaboy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-2151:
-----------------------------------
Resolution: Fixed
Fix Version/s: 0.11
Release Note:
An EvalFunc can annotated with an @OutputSchema annotation to
tell Pig what the expected output is. This can be used in place
of EvalFunc.outputSchema(Schema inpSchema)
The default implementation of EvalFunc.outputSchema(Schema inpSchema) will look at this annotation and return an interpreted schema, if the annotation is present.
Implementing a custom EvalFunc.outputSchema(Schema inpSchema) will override the annotation (unless you deal with it explicitly, or by calling super.outputSchema(schema))
Here's an example of a complex schema declared in an annotation:
@OutputSchema("y:bag{t:tuple(len:int,word:chararray)}")
Status: Resolved (was: Patch Available)
Committed to trunk.
> Add annotation to specify output schema in Java UDFs
> ----------------------------------------------------
>
> Key: PIG-2151
> URL: https://issues.apache.org/jira/browse/PIG-2151
> Project: Pig
> Issue Type: Improvement
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Fix For: 0.11
>
> Attachments: PIG-2151.patch
>
>
> In Python udfs, the output schema of an eval func is specified via an @OutputSchema annotation.
> We should do the same for Java udfs instead of making people write the cumbersome outputSchema functions.
> If an annotation exists, it should override an existing outputSchema implementation (since all evalFuncs will inherit the method from EvalFunc.java and there is thus no way to *not* have an implementation).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2151) Add annotation to specify output schema
in Java UDFs
Posted by "Dmitriy V. Ryaboy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-2151:
-----------------------------------
Attachment: PIG-2151.patch
The attached patch is much more elegant than what I initially proposed -- it simply makes the default implementation of EvalFunc look at the @OutputSchema annotation.
> Add annotation to specify output schema in Java UDFs
> ----------------------------------------------------
>
> Key: PIG-2151
> URL: https://issues.apache.org/jira/browse/PIG-2151
> Project: Pig
> Issue Type: Improvement
> Reporter: Dmitriy V. Ryaboy
> Attachments: PIG-2151.patch
>
>
> In Python udfs, the output schema of an eval func is specified via an @OutputSchema annotation.
> We should do the same for Java udfs instead of making people write the cumbersome outputSchema functions.
> If an annotation exists, it should override an existing outputSchema implementation (since all evalFuncs will inherit the method from EvalFunc.java and there is thus no way to *not* have an implementation).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira