You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Prasad Chakka (JIRA)" <ji...@apache.org> on 2009/06/02 20:47:07 UTC
[jira] Created: (HIVE-533) predicate clauses with rand() should not
be pushed down
predicate clauses with rand() should not be pushed down
-------------------------------------------------------
Key: HIVE-533
URL: https://issues.apache.org/jira/browse/HIVE-533
Project: Hadoop Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.4.0
Reporter: Prasad Chakka
Assignee: Prasad Chakka
in the following query, all predicates except the rand() is pushed down where in rand() is applied after filter which should not be the case.
select a.* from srcpart a where rand(1) < 0.1 and a.ds = '2008-04-08' and not(key > 50 or key < 10) and a.hr like '%2';
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-533) predicate clauses with rand() should
not be pushed down
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729006#action_12729006 ]
Namit Jain commented on HIVE-533:
---------------------------------
Can you add some constants ? like 1/2 for boolean false/true = may help in some debugging later on.
Also, it might be useful to add a dump() method which takes the inputbytebuffer and dumps the structure ?
This can be done in a follow-up later also, may be useful for other serdes also.
> predicate clauses with rand() should not be pushed down
> -------------------------------------------------------
>
> Key: HIVE-533
> URL: https://issues.apache.org/jira/browse/HIVE-533
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Prasad Chakka
> Assignee: Prasad Chakka
> Fix For: 0.4.0
>
> Attachments: hive-533.patch
>
>
> in the following query, all predicates except the rand() is pushed down where in rand() is applied after filter which should not be the case.
> select a.* from srcpart a where rand(1) < 0.1 and a.ds = '2008-04-08' and not(key > 50 or key < 10) and a.hr like '%2';
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-533) predicate clauses with rand() should
not be pushed down
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716060#action_12716060 ]
Namit Jain commented on HIVE-533:
---------------------------------
+1
looks good - will merge if the tests pass
> predicate clauses with rand() should not be pushed down
> -------------------------------------------------------
>
> Key: HIVE-533
> URL: https://issues.apache.org/jira/browse/HIVE-533
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Prasad Chakka
> Assignee: Prasad Chakka
> Fix For: 0.4.0
>
> Attachments: hive-533.patch
>
>
> in the following query, all predicates except the rand() is pushed down where in rand() is applied after filter which should not be the case.
> select a.* from srcpart a where rand(1) < 0.1 and a.ds = '2008-04-08' and not(key > 50 or key < 10) and a.hr like '%2';
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-533) predicate clauses with rand() should not
be pushed down
Posted by "Prasad Chakka (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prasad Chakka updated HIVE-533:
-------------------------------
Fix Version/s: 0.4.0
Status: Patch Available (was: Open)
> predicate clauses with rand() should not be pushed down
> -------------------------------------------------------
>
> Key: HIVE-533
> URL: https://issues.apache.org/jira/browse/HIVE-533
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Prasad Chakka
> Assignee: Prasad Chakka
> Fix For: 0.4.0
>
> Attachments: hive-533.patch
>
>
> in the following query, all predicates except the rand() is pushed down where in rand() is applied after filter which should not be the case.
> select a.* from srcpart a where rand(1) < 0.1 and a.ds = '2008-04-08' and not(key > 50 or key < 10) and a.hr like '%2';
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-533) predicate clauses with rand() should not
be pushed down
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-533:
----------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Committed. Thanks Prasad
> predicate clauses with rand() should not be pushed down
> -------------------------------------------------------
>
> Key: HIVE-533
> URL: https://issues.apache.org/jira/browse/HIVE-533
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Prasad Chakka
> Assignee: Prasad Chakka
> Fix For: 0.4.0
>
> Attachments: hive-533.patch
>
>
> in the following query, all predicates except the rand() is pushed down where in rand() is applied after filter which should not be the case.
> select a.* from srcpart a where rand(1) < 0.1 and a.ds = '2008-04-08' and not(key > 50 or key < 10) and a.hr like '%2';
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-533) predicate clauses with rand() should
not be pushed down
Posted by "Prasad Chakka (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715708#action_12715708 ]
Prasad Chakka commented on HIVE-533:
------------------------------------
we didn't push rand() but we pushed all other predicates in the clause. so rand() is getting applied after rows are filtered. the current behavior seems to be rand() gets applied before rows are filtered. not sure if there is a ANSI semantics, but this fix keeps the current behavior of Hive even when PPD is applied.
> predicate clauses with rand() should not be pushed down
> -------------------------------------------------------
>
> Key: HIVE-533
> URL: https://issues.apache.org/jira/browse/HIVE-533
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Prasad Chakka
> Assignee: Prasad Chakka
>
> in the following query, all predicates except the rand() is pushed down where in rand() is applied after filter which should not be the case.
> select a.* from srcpart a where rand(1) < 0.1 and a.ds = '2008-04-08' and not(key > 50 or key < 10) and a.hr like '%2';
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-533) predicate clauses with rand() should
not be pushed down
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715694#action_12715694 ]
Ashish Thusoo commented on HIVE-533:
------------------------------------
Did we not already handle the case with the non deterministic function support that we had added to predicate pushdown?
> predicate clauses with rand() should not be pushed down
> -------------------------------------------------------
>
> Key: HIVE-533
> URL: https://issues.apache.org/jira/browse/HIVE-533
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Prasad Chakka
> Assignee: Prasad Chakka
>
> in the following query, all predicates except the rand() is pushed down where in rand() is applied after filter which should not be the case.
> select a.* from srcpart a where rand(1) < 0.1 and a.ds = '2008-04-08' and not(key > 50 or key < 10) and a.hr like '%2';
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-533) predicate clauses with rand() should not
be pushed down
Posted by "Prasad Chakka (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prasad Chakka updated HIVE-533:
-------------------------------
Attachment: hive-533.patch
> predicate clauses with rand() should not be pushed down
> -------------------------------------------------------
>
> Key: HIVE-533
> URL: https://issues.apache.org/jira/browse/HIVE-533
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Prasad Chakka
> Assignee: Prasad Chakka
> Fix For: 0.4.0
>
> Attachments: hive-533.patch
>
>
> in the following query, all predicates except the rand() is pushed down where in rand() is applied after filter which should not be the case.
> select a.* from srcpart a where rand(1) < 0.1 and a.ds = '2008-04-08' and not(key > 50 or key < 10) and a.hr like '%2';
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.