You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Zheng Shao (JIRA)" <ji...@apache.org> on 2008/12/11 07:41:44 UTC
[jira] Created: (HIVE-164) Support CASE and IF statement
Support CASE and IF statement
-----------------------------
Key: HIVE-164
URL: https://issues.apache.org/jira/browse/HIVE-164
Project: Hadoop Hive
Issue Type: New Feature
Components: Query Processor
Reporter: Zheng Shao
See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
Syntax:
{code}
CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
IF(expr1,expr2,expr3)
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-164) Support CASE statement using GenericUDF
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-164:
----------------------------
Attachment: HIVE-164.7.patch
Same as HIVE-164.7.code-only.patch but contains test result changes.
> Support CASE statement using GenericUDF
> ---------------------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-164.4.patch, HIVE-164.5.patch, HIVE-164.6.code-only.patch, HIVE-164.6.patch, HIVE-164.7.code-only.patch, HIVE-164.7.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-164) Support CASE statement using
GenericUDF
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12703308#action_12703308 ]
Ashish Thusoo commented on HIVE-164:
------------------------------------
One comment that I want to make by looking at the unit tests is that mysql allows implicit conversion on the comparison expressions. I know that we did not go with the AnyType as the return type of the case statement simply because we do not have the AnyType. However, we should be able to support implicit conversion the comparison expressions. I am fine with that going in as part of a separate JIRA/txn as this one is already quite big and it is important to get this in.
> Support CASE statement using GenericUDF
> ---------------------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-164.4.patch, HIVE-164.5.patch, HIVE-164.6.code-only.patch, HIVE-164.6.patch, HIVE-164.7.code-only.patch, HIVE-164.7.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-164) Support CASE and IF statement
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12668330#action_12668330 ]
Ashish Thusoo commented on HIVE-164:
------------------------------------
Also, once this is checked in please update the wiki docs..
> Support CASE and IF statement
> -----------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-164.1.patch, HIVE-164.2.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> IF(expr1,expr2,expr3)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-164) Support CASE and IF statement
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12668302#action_12668302 ]
Ashish Thusoo commented on HIVE-164:
------------------------------------
Also what are LEFT and RIGHT used for?
> Support CASE and IF statement
> -----------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: David Phillips
> Attachments: HIVE-164.1.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> IF(expr1,expr2,expr3)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-164) Support CASE statement using
GenericUDF
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12704164#action_12704164 ]
Namit Jain commented on HIVE-164:
---------------------------------
+1
I will commit if the tests pass
> Support CASE statement using GenericUDF
> ---------------------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: HIVE-164.4.patch, HIVE-164.5.patch, HIVE-164.6.code-only.patch, HIVE-164.6.patch, HIVE-164.7.code-only.patch, HIVE-164.7.patch, HIVE-164.8.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-164) Support CASE statement using GenericUDF
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-164:
----------------------------
Attachment: (was: HIVE-164.7.patch)
> Support CASE statement using GenericUDF
> ---------------------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-164.4.patch, HIVE-164.5.patch, HIVE-164.6.code-only.patch, HIVE-164.6.patch, HIVE-164.7.code-only.patch, HIVE-164.7.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-164) Support CASE and IF statement
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-164:
----------------------------
Attachment: (was: HIVE-164.2.patch)
> Support CASE and IF statement
> -----------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: Zheng Shao
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> IF(expr1,expr2,expr3)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-164) Support CASE statement using
GenericUDF
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12703310#action_12703310 ]
Ashish Thusoo commented on HIVE-164:
------------------------------------
I would also add a test case where the case expression appears in the where clause along with a partition predicate and a subquery and join. Want to make sure that it gets pushed up properly.
> Support CASE statement using GenericUDF
> ---------------------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-164.4.patch, HIVE-164.5.patch, HIVE-164.6.code-only.patch, HIVE-164.6.patch, HIVE-164.7.code-only.patch, HIVE-164.7.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-164) Support CASE statement using GenericUDF
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-164:
----------------------------
Attachment: HIVE-164.5.patch
Some simple refactoring.
> Support CASE statement using GenericUDF
> ---------------------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-164.4.patch, HIVE-164.5.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-164) Support CASE statement using
GenericUDF
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12700972#action_12700972 ]
Zheng Shao commented on HIVE-164:
---------------------------------
Some framework changes:
* New: GenericUDF API
* ExprNodeEvaluator API changes: init(rowInspector), evaluate(rowObject)
* Operator API change: process() now takes a tag.
Also it would be nice to let the SemanticAnalyzer to use ObjectInspector to represent the row instead of RowSchema.
That can help remove one function from the GenericUDF API.
> Support CASE statement using GenericUDF
> ---------------------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-164.4.patch, HIVE-164.5.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-164) Support CASE and IF statement
Posted by "David Phillips (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656224#action_12656224 ]
David Phillips commented on HIVE-164:
-------------------------------------
I don't understand how UDFTemplate helps here as it doesn't enable the special syntax required for CASE nor does it enable short-circuit evaluation. Implementing CASE as a function would essentially result in Oracle's DECODE:
{{DECODE(expression, search, result [, search , result]... [, default])}}
Even using DECODE() in a simple IF()-like manner can be more difficult to read than CASE, and in more complicated situations it is nearly impossible to understand. I could dig up some real examples...
Short-circuit evaluation is also required:
{{CASE WHEN 1=0 THEN 5/0 ELSE 0 END}}
MySQL supports short-circuit for IF() and Oracle supports it for DECODE(). Doing this in a general way for UDFs seems much more complicated than it's worth. My vote is to support standard CASE and COALESCE() natively. ANSI SQL 92 section 6.9 defines COALESCE():
{noformat}
COALESCE(V1, V2) is equivalent to the following:
CASE WHEN V1 IS NOT NULL THEN V1 ELSE V2 END
COALESCE(V1, V2, ..., Vn), for n >= 3, is equivalent to the following:
CASE WHEN V1 IS NOT NULL THEN V1 ELSE COALESCE(V2, ..., Vn) END
{noformat}
> Support CASE and IF statement
> -----------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> IF(expr1,expr2,expr3)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-164) Support CASE statement using
GenericUDF
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12703309#action_12703309 ]
Ashish Thusoo commented on HIVE-164:
------------------------------------
Also, we should support nulls with other types in the return clause. Basically, udf_case_type_wrong3.q should work.
> Support CASE statement using GenericUDF
> ---------------------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-164.4.patch, HIVE-164.5.patch, HIVE-164.6.code-only.patch, HIVE-164.6.patch, HIVE-164.7.code-only.patch, HIVE-164.7.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-164) Support CASE and IF statement
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12668333#action_12668333 ]
Ashish Thusoo commented on HIVE-164:
------------------------------------
Also we should perhaps do the IF stuff in a separate JIRA or move the collase stuff and the other goodness that David mentioned to a separate JIRA...
David,
Haven't heard from you in a while.. you ok with that?
Ashish
> Support CASE and IF statement
> -----------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-164.1.patch, HIVE-164.2.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> IF(expr1,expr2,expr3)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-164) Support CASE statement using GenericUDF
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashish Thusoo updated HIVE-164:
-------------------------------
Fix Version/s: 0.4.0
Affects Version/s: 0.4.0
Status: Patch Available (was: Open)
Moving to submit patch state.
> Support CASE statement using GenericUDF
> ---------------------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: HIVE-164.4.patch, HIVE-164.5.patch, HIVE-164.6.code-only.patch, HIVE-164.6.patch, HIVE-164.7.code-only.patch, HIVE-164.7.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-164) Support CASE and IF statement
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-164:
----------------------------
Attachment: HIVE-164.1.patch
This simple patch provides "IF".
The majority of the cases are that users just want a simple "IF". This patch provides this simple functionality.
For "CASE" with variable number of arguments, let's move the discussion to a new jira.
> Support CASE and IF statement
> -----------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: David Phillips
> Attachments: HIVE-164.1.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> IF(expr1,expr2,expr3)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-164) Support CASE statement using
GenericUDF
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12703644#action_12703644 ]
Ashish Thusoo commented on HIVE-164:
------------------------------------
+1
Looks fine to me.
Will checkin after running the tests
> Support CASE statement using GenericUDF
> ---------------------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: HIVE-164.4.patch, HIVE-164.5.patch, HIVE-164.6.code-only.patch, HIVE-164.6.patch, HIVE-164.7.code-only.patch, HIVE-164.7.patch, HIVE-164.8.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-164) Support CASE statement using GenericUDF
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-164:
----------------------------
Attachment: HIVE-164.7.patch
This one overwrites a changed test result so all tests are passed.
> Support CASE statement using GenericUDF
> ---------------------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-164.4.patch, HIVE-164.5.patch, HIVE-164.6.code-only.patch, HIVE-164.6.patch, HIVE-164.7.code-only.patch, HIVE-164.7.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-164) Support CASE and IF statement
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656698#action_12656698 ]
Ashish Thusoo commented on HIVE-164:
------------------------------------
+1 to if, case and coalesce
I would also think that these functions would present some challenges in the existing typecheck and expression generation code as these functions are first instance of functions that are nary with n not being fixed.
> Support CASE and IF statement
> -----------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: David Phillips
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> IF(expr1,expr2,expr3)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-164) Support CASE statement using GenericUDF
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-164:
----------------------------
Description:
See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
Syntax:
{code}
CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
{code}
was:
See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
Syntax:
{code}
CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
IF(expr1,expr2,expr3)
{code}
Summary: Support CASE statement using GenericUDF (was: Support CASE and IF statement)
> Support CASE statement using GenericUDF
> ---------------------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-164.4.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-164) Support CASE statement using GenericUDF
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-164:
----------------------------
Attachment: HIVE-164.6.patch
Simplified the interface for GenericUDF. I guess it's easy to write a GenericUDF without too much knowledge of Hive (the developer will still need to know about ObjectInspector).
{code}
@UDFType(deterministic=true)
public abstract class GenericUDF {
/**
* A Defered Object allows us to do lazy-evaluation
* and short-circuiting.
* GenericUDF use DeferedObject to pass arguments.
*/
public static interface DeferredObject {
public Object get() throws HiveException;
};
/**
* The constructor
*/
public GenericUDF() {
}
/**
* Initialize this GenericUDF. This will be called once and only once per
* GenericUDF instance.
*
* @param arguments The ObjectInspector for the arguments
* @throws UDFArgumentTypeException
* Thrown when arguments have wrong types
* @return The ObjectInspector for the return value
*/
public abstract ObjectInspector initialize(ObjectInspector[] arguments)
throws UDFArgumentTypeException;
/**
* Evaluate the GenericUDF with the arguments.
* @param arguments The arguments as DeferedObject, use DeferedObject.get() to
* get the actual argument Object. The Objects can be inspected
* by the ObjectInspectors passed in the initialize call.
* @return The
*/
public abstract Object evaluate(DeferredObject[] arguments) throws HiveException;
/**
* Get the String to be displayed in explain.
*/
public abstract String getDisplayString(String[] children);
}
{code}
> Support CASE statement using GenericUDF
> ---------------------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-164.4.patch, HIVE-164.5.patch, HIVE-164.6.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-164) Support CASE and IF statement
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-164:
----------------------------
Attachment: (was: HIVE-164.1.patch)
> Support CASE and IF statement
> -----------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: Zheng Shao
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> IF(expr1,expr2,expr3)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-164) Support CASE statement using
GenericUDF
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12703350#action_12703350 ]
Namit Jain commented on HIVE-164:
---------------------------------
On the same lines, it would be good to add a testcase where the case expression is in the select clause but does not contain all the columns. Column pruning should still happen
> Support CASE statement using GenericUDF
> ---------------------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-164.4.patch, HIVE-164.5.patch, HIVE-164.6.code-only.patch, HIVE-164.6.patch, HIVE-164.7.code-only.patch, HIVE-164.7.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-164) Support CASE statement using GenericUDF
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-164:
----------------------------
Attachment: HIVE-164.4.patch
Implementation of GenericUDF, and CASE statements using GenericUDF.
> Support CASE statement using GenericUDF
> ---------------------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-164.4.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-164) Support CASE statement using GenericUDF
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-164:
----------------------------
Attachment: HIVE-164.7.code-only.patch
Merged with the latest svn. Code only.
> Support CASE statement using GenericUDF
> ---------------------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-164.4.patch, HIVE-164.5.patch, HIVE-164.6.code-only.patch, HIVE-164.6.patch, HIVE-164.7.code-only.patch, HIVE-164.7.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-164) Support CASE statement using
GenericUDF
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12703959#action_12703959 ]
Zheng Shao commented on HIVE-164:
---------------------------------
HIVE-456 was opened and fixed for the test failure. So this one is ready to go.
> Support CASE statement using GenericUDF
> ---------------------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: HIVE-164.4.patch, HIVE-164.5.patch, HIVE-164.6.code-only.patch, HIVE-164.6.patch, HIVE-164.7.code-only.patch, HIVE-164.7.patch, HIVE-164.8.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-164) Support CASE and IF statement
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656226#action_12656226 ]
Zheng Shao commented on HIVE-164:
---------------------------------
By UDFTemplate, I was refering to the execution part of IF/CASE. IF/CASE/COALESCE can all be considered as UDF except that the type of the arguments can be anything (instead of just PrimitiveTypes like int, string, etc).
In Hive execution code, every object is represented using a pair (o: Object, oi: ObjectInspector). The oi contains information about the type of the object and also how to access the fields/elements inside the object (in case it's a struct/list/map). The interface is exprNodeExprEvaluator.java.
For the grammar part, I think we can support all 3: IF, CASE, COALESCE.
> Support CASE and IF statement
> -----------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: David Phillips
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> IF(expr1,expr2,expr3)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-164) Support CASE statement using GenericUDF
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-164:
----------------------------
Attachment: HIVE-164.8.patch
@Ashish:
Implicit type conversion: Opened HIVE-450.
Nulls: We do allow Nulls: see ql/src/test/queries/clientpositive/udf_case.q. The reason that udf_case_type_wrong3.q does not work is because there is an INT after THEN and a STRING after ELSE.
Partition pruning and predicate push-down: Added ppd_udf_case.q and fixed ppd to handle GenericUDF
{code}
set hive.optimize.ppd=true;
EXPLAIN
SELECT *
FROM srcpart a JOIN srcpart b
ON a.key = b.key
WHERE a.ds = '2008-04-08' AND
b.ds = '2008-04-08' AND
CASE a.key
WHEN '27' THEN TRUE
WHEN '38' THEN FALSE
ELSE NULL
END
ORDER BY a.key, a.value, a.ds, a.hr, b.key, b.value, b.ds, b.hr;
SELECT *
FROM srcpart a JOIN srcpart b
ON a.key = b.key
WHERE a.ds = '2008-04-08' AND
b.ds = '2008-04-08' AND
CASE a.key
WHEN '27' THEN TRUE
WHEN '38' THEN FALSE
ELSE NULL
END
ORDER BY a.key, a.value, a.ds, a.hr, b.key, b.value, b.ds, b.hr;
{code}
@Namit:
Added udf_case_column_pruning.q
{code}
EXPLAIN
SELECT CASE a.key
WHEN '1' THEN 2
WHEN '3' THEN 4
ELSE 5
END as key
FROM src a JOIN src b
ON a.key = b.key
ORDER BY key LIMIT 10;
SELECT CASE a.key
WHEN '1' THEN 2
WHEN '3' THEN 4
ELSE 5
END as key
FROM src a JOIN src b
ON a.key = b.key
ORDER BY key LIMIT 10;
{code}
> Support CASE statement using GenericUDF
> ---------------------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: HIVE-164.4.patch, HIVE-164.5.patch, HIVE-164.6.code-only.patch, HIVE-164.6.patch, HIVE-164.7.code-only.patch, HIVE-164.7.patch, HIVE-164.8.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HIVE-164) Support CASE and IF statement
Posted by "David Phillips (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Phillips reassigned HIVE-164:
-----------------------------------
Assignee: David Phillips
> Support CASE and IF statement
> -----------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: David Phillips
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> IF(expr1,expr2,expr3)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-164) Support CASE and IF statement
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-164:
----------------------------
Attachment: HIVE-164.2.patch
Incorporated comments from Ashish. Reverted change to hive-log4j.properties. Removed LEFT and RIGHT since they are not supported yet.
We don't support Complex types in any UDF functions right now.
> Support CASE and IF statement
> -----------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-164.1.patch, HIVE-164.2.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> IF(expr1,expr2,expr3)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-164) Support CASE and IF statement
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12668305#action_12668305 ]
Ashish Thusoo commented on HIVE-164:
------------------------------------
Also you may want ot add an evaluate method that takes
eval(Boolean, Void, Void)
for the case when both expr2 and expr3 are nulls.
Also what happens in case of complex objects...
> Support CASE and IF statement
> -----------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: David Phillips
> Attachments: HIVE-164.1.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> IF(expr1,expr2,expr3)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-164) Support CASE statement using GenericUDF
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-164:
----------------------------
Attachment: HIVE-164.6.code-only.patch
This is the same as HIVE-164.6.patch but only contains code changes.
> Support CASE statement using GenericUDF
> ---------------------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-164.4.patch, HIVE-164.5.patch, HIVE-164.6.code-only.patch, HIVE-164.6.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-164) Support CASE and IF statement
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12668329#action_12668329 ]
Ashish Thusoo commented on HIVE-164:
------------------------------------
+1 to the new patch.
We should however support complex types. We don't do so mostly because we do not yet have UDFs that have a well defined behavior for complex types. IF clearly has. Can you file a JIRA for that. We can get this in but the other JIRA should go in as a bug fix...
Ashish
PS: Am running tests on this and will commit once they succeed.
> Support CASE and IF statement
> -----------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-164.1.patch, HIVE-164.2.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> IF(expr1,expr2,expr3)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-164) Support CASE statement using
GenericUDF
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12703461#action_12703461 ]
Ashish Thusoo commented on HIVE-164:
------------------------------------
A couple of more things while going over the code..
A question - In most of the xyzOperator.java code the InspectableObjects have been done away with and replaced with ObjectInspector + Object. Why?
I suppose you are going to subsume tag within the row as that makes serialization and deserialization of tag easy. Not sure if you are tackling this as part of HIVE-450. If yes then please ignore this comment.
> Support CASE statement using GenericUDF
> ---------------------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: HIVE-164.4.patch, HIVE-164.5.patch, HIVE-164.6.code-only.patch, HIVE-164.6.patch, HIVE-164.7.code-only.patch, HIVE-164.7.patch, HIVE-164.8.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-164) Support CASE statement using
GenericUDF
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12703687#action_12703687 ]
Ashish Thusoo commented on HIVE-164:
------------------------------------
While running test I am getting out of memory error for union9.q
> Support CASE statement using GenericUDF
> ---------------------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: HIVE-164.4.patch, HIVE-164.5.patch, HIVE-164.6.code-only.patch, HIVE-164.6.patch, HIVE-164.7.code-only.patch, HIVE-164.7.patch, HIVE-164.8.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-164) Support CASE statement using GenericUDF
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-164:
----------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Committed. Thanks Zheng
> Support CASE statement using GenericUDF
> ---------------------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: HIVE-164.4.patch, HIVE-164.5.patch, HIVE-164.6.code-only.patch, HIVE-164.6.patch, HIVE-164.7.code-only.patch, HIVE-164.7.patch, HIVE-164.8.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-164) Support CASE and IF statement
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12668303#action_12668303 ]
Ashish Thusoo commented on HIVE-164:
------------------------------------
The following code has STringLiteral defined twice...
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
===================================================================
--- ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java (revision 738583)
+++ ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java (working copy)
@@ -3293,11 +3293,15 @@
opRules.put(new RuleRegExp("R2", HiveParser.Number + "%"), TypeCheckProcFactory.getNumExprProcessor());
opRules.put(new RuleRegExp("R3", HiveParser.Identifier + "%|" +
HiveParser.StringLiteral + "%|" +
+ HiveParser.KW_IF + "%|" +
+ HiveParser.KW_LEFT + "%|" +
+ HiveParser.KW_RIGHT + "%|" +
+ HiveParser.StringLiteral + "%|" +
HiveParser.TOK_CHARSETLITERAL + "%"),
> Support CASE and IF statement
> -----------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: David Phillips
> Attachments: HIVE-164.1.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> IF(expr1,expr2,expr3)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-164) Support CASE and IF statement
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12668335#action_12668335 ]
Zheng Shao commented on HIVE-164:
---------------------------------
I will move IF to another jira.
> Support CASE and IF statement
> -----------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-164.1.patch, HIVE-164.2.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> IF(expr1,expr2,expr3)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HIVE-164) Support CASE and IF statement
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao reassigned HIVE-164:
-------------------------------
Assignee: Zheng Shao (was: David Phillips)
> Support CASE and IF statement
> -----------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-164.1.patch, HIVE-164.2.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> IF(expr1,expr2,expr3)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-164) Support CASE statement using
GenericUDF
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12703468#action_12703468 ]
Zheng Shao commented on HIVE-164:
---------------------------------
> A question - In most of the xyzOperator.java code the InspectableObjects have been done away with and replaced with ObjectInspector + Object. Why?
Because we don't need InspectableObject there any more - exprNodeEvaluators are dealing with Inspector in initialize and Object in evaluate, so we don't need a wrapper like InspectableObject like before. This takes advantage of the fact that the ObjectInspector that gets passed via InspectableObject is always the same. This also makes the code clearer because developers can directly see that fact. It also improves the performance a little bit because we are doing one less level of indirection.
I don't foresee any places that we would need InspectableObject in the future - it was added as a replacement for HiveObject long time back. Since we are moving the Inspector part to compile time or execution-initialization time, I don't see any places we need to pass Object and ObjectInspector at the same time.
> I suppose you are going to subsume tag within the row as that makes serialization and deserialization of tag easy. Not sure if you are tackling this as part of HIVE-450. If yes then please ignore this comment.
I plan to do that with HIVE-405 since this one has taken too long and introduced a lot of conflicts along the way.
> Support CASE statement using GenericUDF
> ---------------------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: HIVE-164.4.patch, HIVE-164.5.patch, HIVE-164.6.code-only.patch, HIVE-164.6.patch, HIVE-164.7.code-only.patch, HIVE-164.7.patch, HIVE-164.8.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-164) Support CASE statement using
GenericUDF
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12703862#action_12703862 ]
Zheng Shao commented on HIVE-164:
---------------------------------
Error message:
{code}
[junit] Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
[junit] at java.lang.StringCoding$StringDecoder.decode(StringCoding.java:133)
[junit] at java.lang.StringCoding.decode(StringCoding.java:173)
[junit] at java.lang.StringCoding.decode(StringCoding.java:185)
[junit] at java.lang.String.<init>(String.java:571)
[junit] at java.lang.String.<init>(String.java:594)
[junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:448)
[junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
[junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
{code}
The error comes from junit - we are outputting too much stuff to stdout/stderr!
http://www.docjar.com/html/api/org/apache/tools/ant/taskdefs/optional/junit/JUnitTestRunner.java.html
{code}
443 systemError.close();
444 systemError = null;
445 systemOut.close();
446 systemOut = null;
447 if (startTestSuiteSuccess) {
448 sendOutAndErr(new String(outStrm.toByteArray()),
449 new String(errStrm.toByteArray()));
450 }
{code}
I will fix it together with this patch.
> Support CASE statement using GenericUDF
> ---------------------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: HIVE-164.4.patch, HIVE-164.5.patch, HIVE-164.6.code-only.patch, HIVE-164.6.patch, HIVE-164.7.code-only.patch, HIVE-164.7.patch, HIVE-164.8.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-164) Support CASE and IF statement
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12668298#action_12668298 ]
Ashish Thusoo commented on HIVE-164:
------------------------------------
Should the root.logger go back to WARN.DRFA...
Looking at the rest of the diff...
> Support CASE and IF statement
> -----------------------------
>
> Key: HIVE-164
> URL: https://issues.apache.org/jira/browse/HIVE-164
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Zheng Shao
> Assignee: David Phillips
> Attachments: HIVE-164.1.patch
>
>
> See http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html for details.
> Syntax:
> {code}
> CASE value WHEN [compare_value] THEN result [WHEN [compare_value] THEN result ...] [<ELSE result] END
> CASE WHEN [condition] THEN result [WHEN [condition] THEN result ...] [ELSE result] END
> IF(expr1,expr2,expr3)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.