You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Min Zhou (JIRA)" <ji...@apache.org> on 2009/06/10 15:10:07 UTC
[jira] Created: (HIVE-555) create temporary function support not
only udf, but also udaf, genericudf, etc.
create temporary function support not only udf, but also udaf, genericudf, etc.
--------------------------------------------------------------------------------
Key: HIVE-555
URL: https://issues.apache.org/jira/browse/HIVE-555
Project: Hadoop Hive
Issue Type: Improvement
Components: Query Processor
Affects Versions: 0.4.0
Reporter: Min Zhou
Assignee: Min Zhou
Fix For: 0.4.0
Right now, command 'create temporary function' only support udf.
we can also let user write their udaf, generic udf, and write generic udaf in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-555) create temporary function support not
only udf, but also udaf, genericudf, etc.
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12730419#action_12730419 ]
Namit Jain commented on HIVE-555:
---------------------------------
+1
looks good - am running tests right now
> create temporary function support not only udf, but also udaf, genericudf, etc.
> --------------------------------------------------------------------------------
>
> Key: HIVE-555
> URL: https://issues.apache.org/jira/browse/HIVE-555
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Min Zhou
> Assignee: Min Zhou
> Fix For: 0.4.0
>
> Attachments: HIVE-555-1.patch, HIVE-555-2.patch, HIVE-555-3.patch, HIVE-555-4.patch
>
>
> Right now, command 'create temporary function' only support udf.
> we can also let user write their udaf, generic udf, and write generic udaf in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-555) create temporary function support not
only udf, but also udaf, genericudf, etc.
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729322#action_12729322 ]
Namit Jain commented on HIVE-555:
---------------------------------
The tests fail right now.
In the new test create_udaf.q, you need to drop the temporary function test_max.
But a new bug got exposed, if there are two functions pointing to the same class (max and test_max) in this case, the resolver might point to one function even though
another one is referenced. I will test it out separately, and file a jira for that if needed.
Anyway, as this patch stands, all tests after create_udaf.q which reference max, get an error because explain plan shows test_max instead.
> create temporary function support not only udf, but also udaf, genericudf, etc.
> --------------------------------------------------------------------------------
>
> Key: HIVE-555
> URL: https://issues.apache.org/jira/browse/HIVE-555
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Min Zhou
> Assignee: Min Zhou
> Fix For: 0.4.0
>
> Attachments: HIVE-555-1.patch, HIVE-555-2.patch, HIVE-555-3.patch
>
>
> Right now, command 'create temporary function' only support udf.
> we can also let user write their udaf, generic udf, and write generic udaf in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HIVE-555) create temporary function support not
only udf, but also udaf, genericudf, etc.
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain resolved HIVE-555.
-----------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Committed. Thanks Min
> create temporary function support not only udf, but also udaf, genericudf, etc.
> --------------------------------------------------------------------------------
>
> Key: HIVE-555
> URL: https://issues.apache.org/jira/browse/HIVE-555
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Min Zhou
> Assignee: Min Zhou
> Fix For: 0.4.0
>
> Attachments: HIVE-555-1.patch, HIVE-555-2.patch, HIVE-555-3.patch, HIVE-555-4.patch
>
>
> Right now, command 'create temporary function' only support udf.
> we can also let user write their udaf, generic udf, and write generic udaf in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-555) create temporary function support not
only udf, but also udaf, genericudf, etc.
Posted by "Min Zhou (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Min Zhou updated HIVE-555:
--------------------------
Attachment: HIVE-555-2.patch
with unit tests.
> create temporary function support not only udf, but also udaf, genericudf, etc.
> --------------------------------------------------------------------------------
>
> Key: HIVE-555
> URL: https://issues.apache.org/jira/browse/HIVE-555
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Min Zhou
> Assignee: Min Zhou
> Fix For: 0.4.0
>
> Attachments: HIVE-555-1.patch, HIVE-555-2.patch
>
>
> Right now, command 'create temporary function' only support udf.
> we can also let user write their udaf, generic udf, and write generic udaf in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-555) create temporary function support not
only udf, but also udaf, genericudf, etc.
Posted by "Min Zhou (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12728989#action_12728989 ]
Min Zhou commented on HIVE-555:
-------------------------------
1. I thought it would be a common function for generic udf error prompt.
2. It that required for an existing generic udf? but regardless whatever, i'll do it.
> create temporary function support not only udf, but also udaf, genericudf, etc.
> --------------------------------------------------------------------------------
>
> Key: HIVE-555
> URL: https://issues.apache.org/jira/browse/HIVE-555
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Min Zhou
> Assignee: Min Zhou
> Fix For: 0.4.0
>
> Attachments: HIVE-555-1.patch, HIVE-555-2.patch
>
>
> Right now, command 'create temporary function' only support udf.
> we can also let user write their udaf, generic udf, and write generic udaf in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-555) create temporary function support not
only udf, but also udaf, genericudf, etc.
Posted by "Min Zhou (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12730198#action_12730198 ]
Min Zhou commented on HIVE-555:
-------------------------------
@Namit
passed all tests. How is your test there, passed?
> create temporary function support not only udf, but also udaf, genericudf, etc.
> --------------------------------------------------------------------------------
>
> Key: HIVE-555
> URL: https://issues.apache.org/jira/browse/HIVE-555
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Min Zhou
> Assignee: Min Zhou
> Fix For: 0.4.0
>
> Attachments: HIVE-555-1.patch, HIVE-555-2.patch, HIVE-555-3.patch, HIVE-555-4.patch
>
>
> Right now, command 'create temporary function' only support udf.
> we can also let user write their udaf, generic udf, and write generic udaf in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-555) create temporary function support not
only udf, but also udaf, genericudf, etc.
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729673#action_12729673 ]
Namit Jain commented on HIVE-555:
---------------------------------
But you need to add that in order for the tests to work.
Or, file a new jira for that and change your tests to use a existing class.
And, please run all tests before filing a new patch.
> create temporary function support not only udf, but also udaf, genericudf, etc.
> --------------------------------------------------------------------------------
>
> Key: HIVE-555
> URL: https://issues.apache.org/jira/browse/HIVE-555
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Min Zhou
> Assignee: Min Zhou
> Fix For: 0.4.0
>
> Attachments: HIVE-555-1.patch, HIVE-555-2.patch, HIVE-555-3.patch
>
>
> Right now, command 'create temporary function' only support udf.
> we can also let user write their udaf, generic udf, and write generic udaf in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-555) create temporary function support not
only udf, but also udaf, genericudf, etc.
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729004#action_12729004 ]
Zheng Shao commented on HIVE-555:
---------------------------------
@HIVE-555-3.patch.
Can we change the error messages to something like "Class xxxx not found" or "Class xxxx does not implement UDF, GenericUDF, or UDAF" ?
> create temporary function support not only udf, but also udaf, genericudf, etc.
> --------------------------------------------------------------------------------
>
> Key: HIVE-555
> URL: https://issues.apache.org/jira/browse/HIVE-555
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Min Zhou
> Assignee: Min Zhou
> Fix For: 0.4.0
>
> Attachments: HIVE-555-1.patch, HIVE-555-2.patch, HIVE-555-3.patch
>
>
> Right now, command 'create temporary function' only support udf.
> we can also let user write their udaf, generic udf, and write generic udaf in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-555) create temporary function support not
only udf, but also udaf, genericudf, etc.
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729332#action_12729332 ]
Namit Jain commented on HIVE-555:
---------------------------------
Confirmed the issue above: filed a new jira for that
http://issues.apache.org/jira/browse/HIVE-620
This is an existing bug and can be resolved later. But you need to modify your test case to drop test_max at the end so that tests can pass
> create temporary function support not only udf, but also udaf, genericudf, etc.
> --------------------------------------------------------------------------------
>
> Key: HIVE-555
> URL: https://issues.apache.org/jira/browse/HIVE-555
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Min Zhou
> Assignee: Min Zhou
> Fix For: 0.4.0
>
> Attachments: HIVE-555-1.patch, HIVE-555-2.patch, HIVE-555-3.patch
>
>
> Right now, command 'create temporary function' only support udf.
> we can also let user write their udaf, generic udf, and write generic udaf in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-555) create temporary function support not
only udf, but also udaf, genericudf, etc.
Posted by "Min Zhou (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Min Zhou updated HIVE-555:
--------------------------
Attachment: HIVE-555-3.patch
patch followed namit's comments.
> create temporary function support not only udf, but also udaf, genericudf, etc.
> --------------------------------------------------------------------------------
>
> Key: HIVE-555
> URL: https://issues.apache.org/jira/browse/HIVE-555
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Min Zhou
> Assignee: Min Zhou
> Fix For: 0.4.0
>
> Attachments: HIVE-555-1.patch, HIVE-555-2.patch, HIVE-555-3.patch
>
>
> Right now, command 'create temporary function' only support udf.
> we can also let user write their udaf, generic udf, and write generic udaf in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-555) create temporary function support not
only udf, but also udaf, genericudf, etc.
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729305#action_12729305 ]
Namit Jain commented on HIVE-555:
---------------------------------
Agreed, that is a separate problem which also exists currently for UDFs. Filed a separate jira for that
https://issues.apache.org/jira/browse/HIVE-619
+1
Running tests right now - will commit if the tests pass
> create temporary function support not only udf, but also udaf, genericudf, etc.
> --------------------------------------------------------------------------------
>
> Key: HIVE-555
> URL: https://issues.apache.org/jira/browse/HIVE-555
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Min Zhou
> Assignee: Min Zhou
> Fix For: 0.4.0
>
> Attachments: HIVE-555-1.patch, HIVE-555-2.patch, HIVE-555-3.patch
>
>
> Right now, command 'create temporary function' only support udf.
> we can also let user write their udaf, generic udf, and write generic udaf in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-555) create temporary function support not
only udf, but also udaf, genericudf, etc.
Posted by "Min Zhou (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729502#action_12729502 ]
Min Zhou commented on HIVE-555:
-------------------------------
I guess Hive currently haven't any facilities for droping a temporary function.
> create temporary function support not only udf, but also udaf, genericudf, etc.
> --------------------------------------------------------------------------------
>
> Key: HIVE-555
> URL: https://issues.apache.org/jira/browse/HIVE-555
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Min Zhou
> Assignee: Min Zhou
> Fix For: 0.4.0
>
> Attachments: HIVE-555-1.patch, HIVE-555-2.patch, HIVE-555-3.patch
>
>
> Right now, command 'create temporary function' only support udf.
> we can also let user write their udaf, generic udf, and write generic udaf in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-555) create temporary function support not
only udf, but also udaf, genericudf, etc.
Posted by "Min Zhou (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729009#action_12729009 ]
Min Zhou commented on HIVE-555:
-------------------------------
@Zheng
It would involved some logic out of the FuctionTask. Actually , execute methods of all Task classes is defined to return an integer stand for status code. So create another jira for that issue is better. Agree?
> create temporary function support not only udf, but also udaf, genericudf, etc.
> --------------------------------------------------------------------------------
>
> Key: HIVE-555
> URL: https://issues.apache.org/jira/browse/HIVE-555
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Min Zhou
> Assignee: Min Zhou
> Fix For: 0.4.0
>
> Attachments: HIVE-555-1.patch, HIVE-555-2.patch, HIVE-555-3.patch
>
>
> Right now, command 'create temporary function' only support udf.
> we can also let user write their udaf, generic udf, and write generic udaf in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-555) create temporary function support not
only udf, but also udaf, genericudf, etc.
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12728921#action_12728921 ]
Namit Jain commented on HIVE-555:
---------------------------------
Mostly looks good - I had a couple of minor comments.
1. Can you move
+ /**
+ * Return a ordianl from an integer.
+ */
+ public static String getOrdinal(int i) {
+ int unit = i % 10;
+ return (i <= 0) ? ""
+ : (i != 11 && unit == 1) ? i + "st"
+ : (i != 12 && unit == 2) ? i + "nd"
+ : (i != 13 && unit == 3) ? i + "rd"
+ : i + "th";
+ }
to the test itself instead of GenericUDFUtils
(change ordianl to ordinal)
2. add a test which creates a temporary function on a existing generic udf.
3. add a negative test - the class does not exist
> create temporary function support not only udf, but also udaf, genericudf, etc.
> --------------------------------------------------------------------------------
>
> Key: HIVE-555
> URL: https://issues.apache.org/jira/browse/HIVE-555
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Min Zhou
> Assignee: Min Zhou
> Fix For: 0.4.0
>
> Attachments: HIVE-555-1.patch, HIVE-555-2.patch
>
>
> Right now, command 'create temporary function' only support udf.
> we can also let user write their udaf, generic udf, and write generic udaf in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-555) create temporary function support not
only udf, but also udaf, genericudf, etc.
Posted by "Min Zhou (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Min Zhou updated HIVE-555:
--------------------------
Attachment: HIVE-555-1.patch
patch w/o testcase
> create temporary function support not only udf, but also udaf, genericudf, etc.
> --------------------------------------------------------------------------------
>
> Key: HIVE-555
> URL: https://issues.apache.org/jira/browse/HIVE-555
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Min Zhou
> Assignee: Min Zhou
> Fix For: 0.4.0
>
> Attachments: HIVE-555-1.patch
>
>
> Right now, command 'create temporary function' only support udf.
> we can also let user write their udaf, generic udf, and write generic udaf in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-555) create temporary function support not
only udf, but also udaf, genericudf, etc.
Posted by "Min Zhou (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Min Zhou updated HIVE-555:
--------------------------
Attachment: HIVE-555-4.patch
Add a copy of UDAF to avoid [HIVE-620|http://issues.apache.org/jira/browse/HIVE-620] for passing all test cases.
> create temporary function support not only udf, but also udaf, genericudf, etc.
> --------------------------------------------------------------------------------
>
> Key: HIVE-555
> URL: https://issues.apache.org/jira/browse/HIVE-555
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Min Zhou
> Assignee: Min Zhou
> Fix For: 0.4.0
>
> Attachments: HIVE-555-1.patch, HIVE-555-2.patch, HIVE-555-3.patch, HIVE-555-4.patch
>
>
> Right now, command 'create temporary function' only support udf.
> we can also let user write their udaf, generic udf, and write generic udaf in the future.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.