You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Jonathan Chang (Created) (JIRA)" <ji...@apache.org> on 2011/10/28 18:47:32 UTC
[jira] [Created] (HIVE-2532) Evaluation of
non-deterministic/stateful UDFs should not be skipped even if constant oi
is returned.
Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.
----------------------------------------------------------------------------------------------------
Key: HIVE-2532
URL: https://issues.apache.org/jira/browse/HIVE-2532
Project: Hive
Issue Type: Bug
Reporter: Jonathan Chang
Assignee: Jonathan Chang
Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2532) Evaluation of
non-deterministic/stateful UDFs should not be skipped even if constant oi
is returned.
Posted by "John Sichi (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138628#comment-13138628 ]
John Sichi commented on HIVE-2532:
----------------------------------
contrib/src/java/org/apache/hadoop/hive/contrib/udf/UDFRowSequence.java
(I used this for the existing stateful test cases.)
> Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2532
> URL: https://issues.apache.org/jira/browse/HIVE-2532
> Project: Hive
> Issue Type: Bug
> Reporter: Jonathan Chang
> Assignee: Jonathan Chang
>
> Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2532) Evaluation of
non-deterministic/stateful UDFs should not be skipped even if constant oi
is returned.
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148074#comment-13148074 ]
Phabricator commented on HIVE-2532:
-----------------------------------
jsichi has commented on the revision "HIVE-2532 [jira] Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.".
INLINE COMMENTS
ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeGenericFuncEvaluator.java:144 & is eager; && short-circuits
REVISION DETAIL
https://reviews.facebook.net/D273
> Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2532
> URL: https://issues.apache.org/jira/browse/HIVE-2532
> Project: Hive
> Issue Type: Bug
> Reporter: Jonathan Chang
> Assignee: Jonathan Chang
> Attachments: D273.1.patch, D273.2.patch, D273.3.patch
>
>
> Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2532) Evaluation of
non-deterministic/stateful UDFs should not be skipped even if constant oi
is returned.
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2532:
------------------------------
Attachment: D273.2.patch
jonchang updated the revision "HIVE-2532 [jira] Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.".
Reviewers: JIRA, jsichi
Make deterministic check recursive. Add unittest to test for this case.
REVISION DETAIL
https://reviews.facebook.net/D273
AFFECTED FILES
ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeEvaluator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeGenericFuncEvaluator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAssertTrue.java
ql/src/test/queries/clientnegative/udf_assert_true.q
ql/src/test/queries/clientnegative/udf_assert_true2.q
ql/src/test/results/clientnegative/udf_assert_true.q.out
ql/src/test/results/clientnegative/udf_assert_true2.q.out
ql/src/test/results/clientpositive/show_functions.q.out
> Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2532
> URL: https://issues.apache.org/jira/browse/HIVE-2532
> Project: Hive
> Issue Type: Bug
> Reporter: Jonathan Chang
> Assignee: Jonathan Chang
> Attachments: D273.1.patch, D273.2.patch
>
>
> Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2532) Evaluation of
non-deterministic/stateful UDFs should not be skipped even if constant oi
is returned.
Posted by "rohithsharma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427133#comment-13427133 ]
rohithsharma commented on HIVE-2532:
------------------------------------
Hi,
I have one doubt on ASSERT_TRUE() funtion. Currently ASSERT_TRUE() is returning NULL.Its PrimitiveType is VOID.Once query is executed in CLI mode, output of query prints NULL in console where as in JDBC client throw an exception(HIVE-3200).The serde implimentation for PrimitiveType VOID throw exception.
Correct me if I am wrong,
>From the user perspective I feel its better to return boolean value i.e TRUE instead of returning NULL. Please give your opinion..
> Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2532
> URL: https://issues.apache.org/jira/browse/HIVE-2532
> Project: Hive
> Issue Type: Bug
> Reporter: Jonathan Chang
> Assignee: Jonathan Chang
> Fix For: 0.8.0
>
> Attachments: D273.1.patch, D273.2.patch, D273.3.patch, D273.4.patch, D273.4.patch, D273.4.patch
>
>
> Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2532) Evaluation of
non-deterministic/stateful UDFs should not be skipped even if constant oi
is returned.
Posted by "Jonathan Chang (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Chang updated HIVE-2532:
---------------------------------
Attachment: D273.4.patch
> Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2532
> URL: https://issues.apache.org/jira/browse/HIVE-2532
> Project: Hive
> Issue Type: Bug
> Reporter: Jonathan Chang
> Assignee: Jonathan Chang
> Attachments: D273.1.patch, D273.2.patch, D273.3.patch, D273.4.patch, D273.4.patch, D273.4.patch
>
>
> Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2532) Evaluation of
non-deterministic/stateful UDFs should not be skipped even if constant oi
is returned.
Posted by "Jonathan Chang (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Chang updated HIVE-2532:
---------------------------------
Attachment: D273.4.patch
> Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2532
> URL: https://issues.apache.org/jira/browse/HIVE-2532
> Project: Hive
> Issue Type: Bug
> Reporter: Jonathan Chang
> Assignee: Jonathan Chang
> Attachments: D273.1.patch, D273.2.patch, D273.3.patch, D273.4.patch, D273.4.patch
>
>
> Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2532) Evaluation of
non-deterministic/stateful UDFs should not be skipped even if constant oi
is returned.
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148687#comment-13148687 ]
Phabricator commented on HIVE-2532:
-----------------------------------
jsichi has accepted the revision "HIVE-2532 [jira] Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.".
spockwin
REVISION DETAIL
https://reviews.facebook.net/D273
> Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2532
> URL: https://issues.apache.org/jira/browse/HIVE-2532
> Project: Hive
> Issue Type: Bug
> Reporter: Jonathan Chang
> Assignee: Jonathan Chang
> Attachments: D273.1.patch, D273.2.patch, D273.3.patch, D273.4.patch
>
>
> Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2532) Evaluation of
non-deterministic/stateful UDFs should not be skipped even if constant oi
is returned.
Posted by "Jonathan Chang (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138599#comment-13138599 ]
Jonathan Chang commented on HIVE-2532:
--------------------------------------
Ooo, how about a counter diff?
> Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2532
> URL: https://issues.apache.org/jira/browse/HIVE-2532
> Project: Hive
> Issue Type: Bug
> Reporter: Jonathan Chang
> Assignee: Jonathan Chang
>
> Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2532) Evaluation of
non-deterministic/stateful UDFs should not be skipped even if constant oi
is returned.
Posted by "rohithsharma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427249#comment-13427249 ]
rohithsharma commented on HIVE-2532:
------------------------------------
I tested with changing implementation to boolean.All the positive scenario works fine. Whereas , "SELECT 1 + assert_true FROM src" fails.
{noformat}
2012-08-02 16:34:31,173 ERROR ql.Driver (SessionState.java:printError(400)) - FAILED: Error in semantic analysis: Line 1:7 Wrong arguments '0': Unable to find a common class betweentypes int and boolean
org.apache.hadoop.hive.ql.parse.SemanticException: Line 1:7 Wrong arguments '0': Unable to find a common class betweentypes int and boolean
at org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(TypeCheckProcFactory.java:896)
at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:89)
{noformat}
With string return type, "SELECT 1 + assert_true FROM src" query execution is successfull.
> Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2532
> URL: https://issues.apache.org/jira/browse/HIVE-2532
> Project: Hive
> Issue Type: Bug
> Reporter: Jonathan Chang
> Assignee: Jonathan Chang
> Fix For: 0.8.0
>
> Attachments: D273.1.patch, D273.2.patch, D273.3.patch, D273.4.patch, D273.4.patch, D273.4.patch
>
>
> Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2532) Evaluation of
non-deterministic/stateful UDFs should not be skipped even if constant oi
is returned.
Posted by "Jonathan Chang (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138595#comment-13138595 ]
Jonathan Chang commented on HIVE-2532:
--------------------------------------
I have a patch but don't have a good testcase yet. This might be a good time to add a side-effecting UDF. Any suggestions?
> Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2532
> URL: https://issues.apache.org/jira/browse/HIVE-2532
> Project: Hive
> Issue Type: Bug
> Reporter: Jonathan Chang
> Assignee: Jonathan Chang
>
> Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2532) Evaluation of
non-deterministic/stateful UDFs should not be skipped even if constant oi
is returned.
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2532:
------------------------------
Attachment: D273.1.patch
jonchang requested code review of "HIVE-2532 [jira] Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.".
Reviewers: JIRA
HIVE-2532
Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
TEST PLAN
EMPTY
REVISION DETAIL
https://reviews.facebook.net/D273
AFFECTED FILES
ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeGenericFuncEvaluator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAssertTrue.java
ql/src/test/queries/clientnegative/udf_assert_true.q
ql/src/test/results/clientnegative/udf_assert_true.q.out
ql/src/test/results/clientpositive/show_functions.q.out
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/519/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
> Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2532
> URL: https://issues.apache.org/jira/browse/HIVE-2532
> Project: Hive
> Issue Type: Bug
> Reporter: Jonathan Chang
> Assignee: Jonathan Chang
> Attachments: D273.1.patch
>
>
> Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2532) Evaluation of
non-deterministic/stateful UDFs should not be skipped even if constant oi
is returned.
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148699#comment-13148699 ]
Phabricator commented on HIVE-2532:
-----------------------------------
jonchang has committed the revision "HIVE-2532 [jira] Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.".
REVISION DETAIL
https://reviews.facebook.net/D273
COMMIT
https://reviews.facebook.net/rHIVE1201035
> Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2532
> URL: https://issues.apache.org/jira/browse/HIVE-2532
> Project: Hive
> Issue Type: Bug
> Reporter: Jonathan Chang
> Assignee: Jonathan Chang
> Attachments: D273.1.patch, D273.2.patch, D273.3.patch, D273.4.patch, D273.4.patch, D273.4.patch
>
>
> Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2532) Evaluation of
non-deterministic/stateful UDFs should not be skipped even if constant oi
is returned.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148861#comment-13148861 ]
Hudson commented on HIVE-2532:
------------------------------
Integrated in Hive-trunk-h0.21 #1078 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1078/])
HIVE-2532 [jira] Evaluation of non-deterministic/stateful UDFs should not be
skipped even if constant oi is returned.
(Jonathan Chang via jvs)
Summary:
HIVE-2532
Even if constant oi is returned, these may have stateful/side-effect behavior
and hence need to be called each cycle.
Test Plan: EMPTY
Reviewers: JIRA, jsichi
Reviewed By: jsichi
CC: ikabiljo, jonchang, jsichi
Differential Revision: 273
jvs : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201035
Files :
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeEvaluator.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeGenericFuncEvaluator.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAssertTrue.java
* /hive/trunk/ql/src/test/queries/clientnegative/udf_assert_true.q
* /hive/trunk/ql/src/test/queries/clientnegative/udf_assert_true2.q
* /hive/trunk/ql/src/test/results/clientnegative/udf_assert_true.q.out
* /hive/trunk/ql/src/test/results/clientnegative/udf_assert_true2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/show_functions.q.out
> Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2532
> URL: https://issues.apache.org/jira/browse/HIVE-2532
> Project: Hive
> Issue Type: Bug
> Reporter: Jonathan Chang
> Assignee: Jonathan Chang
> Fix For: 0.9.0
>
> Attachments: D273.1.patch, D273.2.patch, D273.3.patch, D273.4.patch, D273.4.patch, D273.4.patch
>
>
> Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2532) Evaluation of
non-deterministic/stateful UDFs should not be skipped even if constant oi
is returned.
Posted by "Igor Kabiljo (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139367#comment-13139367 ]
Igor Kabiljo commented on HIVE-2532:
------------------------------------
@John: UDFRowSequence should already work, problem is when UDF returns ConstantOI, but has side-effects.
One (not very useful) pair of UDFs for this could be FB_LOCAL_PROPERTY_SET(key, value) and FB_LOCAL_PROPERTY_GET(key), where FB_LOCAL_PROPERTY_SET returns ConstVoidOI, but has side-effects
> Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2532
> URL: https://issues.apache.org/jira/browse/HIVE-2532
> Project: Hive
> Issue Type: Bug
> Reporter: Jonathan Chang
> Assignee: Jonathan Chang
>
> Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2532) Evaluation of
non-deterministic/stateful UDFs should not be skipped even if constant oi
is returned.
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2532:
------------------------------
Attachment: D273.4.patch
jonchang updated the revision "HIVE-2532 [jira] Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.".
Reviewers: JIRA, jsichi
& -> &&
REVISION DETAIL
https://reviews.facebook.net/D273
AFFECTED FILES
ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeEvaluator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeGenericFuncEvaluator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAssertTrue.java
ql/src/test/queries/clientnegative/udf_assert_true.q
ql/src/test/queries/clientnegative/udf_assert_true2.q
ql/src/test/results/clientnegative/udf_assert_true.q.out
ql/src/test/results/clientnegative/udf_assert_true2.q.out
ql/src/test/results/clientpositive/show_functions.q.out
> Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2532
> URL: https://issues.apache.org/jira/browse/HIVE-2532
> Project: Hive
> Issue Type: Bug
> Reporter: Jonathan Chang
> Assignee: Jonathan Chang
> Attachments: D273.1.patch, D273.2.patch, D273.3.patch, D273.4.patch
>
>
> Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2532) Evaluation of
non-deterministic/stateful UDFs should not be skipped even if constant oi
is returned.
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13146526#comment-13146526 ]
Phabricator commented on HIVE-2532:
-----------------------------------
jsichi has commented on the revision "HIVE-2532 [jira] Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.".
Try this:
SELECT 1+ASSERT_TRUE(x < 2) FROM src LATERAL VIEW EXPLODE(ARRAY(1, 2)) a AS x LIMIT 2;
I assume it should hit an exception, but it actually passes. Guess why?
INLINE COMMENTS
ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeGenericFuncEvaluator.java:144 FunctionRegistry.isDeterministic always returns false for a stateful UDF, so you don't need to double-check here.
REVISION DETAIL
https://reviews.facebook.net/D273
> Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2532
> URL: https://issues.apache.org/jira/browse/HIVE-2532
> Project: Hive
> Issue Type: Bug
> Reporter: Jonathan Chang
> Assignee: Jonathan Chang
> Attachments: D273.1.patch
>
>
> Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2532) Evaluation of
non-deterministic/stateful UDFs should not be skipped even if constant oi
is returned.
Posted by "John Sichi (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148686#comment-13148686 ]
John Sichi commented on HIVE-2532:
----------------------------------
+1, tests passed. Can you grant ASF rights?
> Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2532
> URL: https://issues.apache.org/jira/browse/HIVE-2532
> Project: Hive
> Issue Type: Bug
> Reporter: Jonathan Chang
> Assignee: Jonathan Chang
> Attachments: D273.1.patch, D273.2.patch, D273.3.patch, D273.4.patch
>
>
> Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2532) Evaluation of
non-deterministic/stateful UDFs should not be skipped even if constant oi
is returned.
Posted by "Jonathan Chang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427197#comment-13427197 ]
Jonathan Chang commented on HIVE-2532:
--------------------------------------
It's ok by me if you want to return a bool instead. That being said, things should not crash if things are NULL (I think this was a recent regression and there's a JIRA open for it).
> Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2532
> URL: https://issues.apache.org/jira/browse/HIVE-2532
> Project: Hive
> Issue Type: Bug
> Reporter: Jonathan Chang
> Assignee: Jonathan Chang
> Fix For: 0.8.0
>
> Attachments: D273.1.patch, D273.2.patch, D273.3.patch, D273.4.patch, D273.4.patch, D273.4.patch
>
>
> Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2532) Evaluation of
non-deterministic/stateful UDFs should not be skipped even if constant oi
is returned.
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144883#comment-13144883 ]
Phabricator commented on HIVE-2532:
-----------------------------------
jonchang has commented on the revision "HIVE-2532 [jira] Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.".
INLINE COMMENTS
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAssertTrue.java:60-61 Boolean is not really much less difficult and someone once told me that the Writables are more efficient.
REVISION DETAIL
https://reviews.facebook.net/D273
> Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2532
> URL: https://issues.apache.org/jira/browse/HIVE-2532
> Project: Hive
> Issue Type: Bug
> Reporter: Jonathan Chang
> Assignee: Jonathan Chang
> Attachments: D273.1.patch
>
>
> Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2532) Evaluation of
non-deterministic/stateful UDFs should not be skipped even if constant oi
is returned.
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144875#comment-13144875 ]
Phabricator commented on HIVE-2532:
-----------------------------------
ikabiljo has commented on the revision "HIVE-2532 [jira] Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.".
Java code looks good, not sure what are .q and .q.out files, so will leave to others to accept :)
Just a small inline comment.
INLINE COMMENTS
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAssertTrue.java:60-61 why don't convert to javaBooleanObjectInspector, usage is easier?
REVISION DETAIL
https://reviews.facebook.net/D273
> Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2532
> URL: https://issues.apache.org/jira/browse/HIVE-2532
> Project: Hive
> Issue Type: Bug
> Reporter: Jonathan Chang
> Assignee: Jonathan Chang
> Attachments: D273.1.patch
>
>
> Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-2532) Evaluation of
non-deterministic/stateful UDFs should not be skipped even if constant oi
is returned.
Posted by "John Sichi (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi resolved HIVE-2532.
------------------------------
Resolution: Fixed
Fix Version/s: 0.9.0
Hadoop Flags: Reviewed
Committed to trunk, thanks Jonathan! Also don't forget to update the wiki.
> Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2532
> URL: https://issues.apache.org/jira/browse/HIVE-2532
> Project: Hive
> Issue Type: Bug
> Reporter: Jonathan Chang
> Assignee: Jonathan Chang
> Fix For: 0.9.0
>
> Attachments: D273.1.patch, D273.2.patch, D273.3.patch, D273.4.patch, D273.4.patch, D273.4.patch
>
>
> Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2532) Evaluation of
non-deterministic/stateful UDFs should not be skipped even if constant oi
is returned.
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13147988#comment-13147988 ]
Phabricator commented on HIVE-2532:
-----------------------------------
jsichi has requested changes to the revision "HIVE-2532 [jira] Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.".
INLINE COMMENTS
ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeGenericFuncEvaluator.java:144 Note that Java does not short-circuit the &= assignment.
http://stackoverflow.com/questions/3152208/do-and-short-circuit-in-java
ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeEvaluator.java:43 Need Javadoc here.
REVISION DETAIL
https://reviews.facebook.net/D273
> Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2532
> URL: https://issues.apache.org/jira/browse/HIVE-2532
> Project: Hive
> Issue Type: Bug
> Reporter: Jonathan Chang
> Assignee: Jonathan Chang
> Attachments: D273.1.patch, D273.2.patch
>
>
> Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2532) Evaluation of
non-deterministic/stateful UDFs should not be skipped even if constant oi
is returned.
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148096#comment-13148096 ]
Phabricator commented on HIVE-2532:
-----------------------------------
jsichi has commented on the revision "HIVE-2532 [jira] Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.".
OK I'll kick off tests.
REVISION DETAIL
https://reviews.facebook.net/D273
> Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2532
> URL: https://issues.apache.org/jira/browse/HIVE-2532
> Project: Hive
> Issue Type: Bug
> Reporter: Jonathan Chang
> Assignee: Jonathan Chang
> Attachments: D273.1.patch, D273.2.patch, D273.3.patch, D273.4.patch
>
>
> Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2532) Evaluation of
non-deterministic/stateful UDFs should not be skipped even if constant oi
is returned.
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2532:
------------------------------
Attachment: D273.3.patch
jonchang updated the revision "HIVE-2532 [jira] Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.".
Reviewers: JIRA, jsichi
jvs' comments.
REVISION DETAIL
https://reviews.facebook.net/D273
AFFECTED FILES
ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeEvaluator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeGenericFuncEvaluator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAssertTrue.java
ql/src/test/queries/clientnegative/udf_assert_true.q
ql/src/test/queries/clientnegative/udf_assert_true2.q
ql/src/test/results/clientnegative/udf_assert_true.q.out
ql/src/test/results/clientnegative/udf_assert_true2.q.out
ql/src/test/results/clientpositive/show_functions.q.out
> Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2532
> URL: https://issues.apache.org/jira/browse/HIVE-2532
> Project: Hive
> Issue Type: Bug
> Reporter: Jonathan Chang
> Assignee: Jonathan Chang
> Attachments: D273.1.patch, D273.2.patch, D273.3.patch
>
>
> Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2532) Evaluation of
non-deterministic/stateful UDFs should not be skipped even if constant oi
is returned.
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148073#comment-13148073 ]
Phabricator commented on HIVE-2532:
-----------------------------------
jonchang has commented on the revision "HIVE-2532 [jira] Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.".
INLINE COMMENTS
ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeGenericFuncEvaluator.java:144 Really? Jeebus.
REVISION DETAIL
https://reviews.facebook.net/D273
> Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2532
> URL: https://issues.apache.org/jira/browse/HIVE-2532
> Project: Hive
> Issue Type: Bug
> Reporter: Jonathan Chang
> Assignee: Jonathan Chang
> Attachments: D273.1.patch, D273.2.patch, D273.3.patch
>
>
> Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2532) Evaluation of
non-deterministic/stateful UDFs should not be skipped even if constant oi
is returned.
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13146562#comment-13146562 ]
Phabricator commented on HIVE-2532:
-----------------------------------
jonchang has planned changes to the revision "HIVE-2532 [jira] Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.".
Ugh. Will fix and add new unittest
REVISION DETAIL
https://reviews.facebook.net/D273
> Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2532
> URL: https://issues.apache.org/jira/browse/HIVE-2532
> Project: Hive
> Issue Type: Bug
> Reporter: Jonathan Chang
> Assignee: Jonathan Chang
> Attachments: D273.1.patch
>
>
> Even if constant oi is returned, these may have stateful/side-effect behavior and hence need to be called each cycle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira