You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Navis (Created) (JIRA)" <ji...@apache.org> on 2012/04/04 06:09:21 UTC
[jira] [Created] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Support non-MR fetching for simple queries with select/limit/filter operations only
-----------------------------------------------------------------------------------
Key: HIVE-2925
URL: https://issues.apache.org/jira/browse/HIVE-2925
Project: Hive
Issue Type: Improvement
Reporter: Navis
Assignee: Navis
Priority: Trivial
It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-2925:
------------------------
Status: Patch Available (was: Open)
Rebased on trunk. Sorry for late reply.
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.1.patch.txt, HIVE-2925.2.patch.txt, HIVE-2925.3.patch.txt, HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13423742#comment-13423742 ]
Namit Jain commented on HIVE-2925:
----------------------------------
Sorry to have missed this.
Will take a look now.
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-2925:
------------------------
Attachment: HIVE-2925.1.patch.txt
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.1.patch.txt, HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13428571#comment-13428571 ]
Namit Jain commented on HIVE-2925:
----------------------------------
To validate this, can you also add equivalent of partition_wife_fileformat3.q with "more" non-mr fetch.
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.1.patch.txt, HIVE-2925.2.patch.txt, HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-2925:
------------------------
Status: Patch Available (was: Open)
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274782#comment-13274782 ]
Phabricator commented on HIVE-2925:
-----------------------------------
njain has commented on the revision "HIVE-2925 [jira] Support non-MR fetching for simple queries with select/limit/filter operations only".
INLINE COMMENTS
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:6964 Do you want to push the all these checks also in isSimpleSelectQuery ?
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:6976 empty where clause should be checked in isSelectStarOnly ?
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:6974 No need to initialize work if 1 of the below 2 conds is not true
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:7030 I am missing something here: can the code be refactored so that this check happens only
at the beginning ?
I know, the code is not clean, but can you do some cleanup here ?
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:7346 This is due to legacy bad code - now that you are cleaning it, it would be much simpler
if all this was moved to a new optimizer step (including select star optimizations).
That would simplify the code a lot, and also makes it easier to disable this optimization.
ql/src/java/org/apache/hadoop/hive/ql/exec/FetchTask.java:135 Do you need this ? Can you use FileSinkOperator ?
REVISION DETAIL
https://reviews.facebook.net/D2607
To: JIRA, navis
Cc: njain
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13276420#comment-13276420 ]
Phabricator commented on HIVE-2925:
-----------------------------------
navis has commented on the revision "HIVE-2925 [jira] Support non-MR fetching for simple queries with select/limit/filter operations only".
Ok, I'll try in next week.
REVISION DETAIL
https://reviews.facebook.net/D2607
To: JIRA, navis
Cc: njain
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-2925:
------------------------
Attachment: HIVE-2925.3.patch.txt
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.1.patch.txt, HIVE-2925.2.patch.txt, HIVE-2925.3.patch.txt, HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-2925:
-----------------------------
Status: Open (was: Patch Available)
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427156#comment-13427156 ]
Navis commented on HIVE-2925:
-----------------------------
dummy assignment removal and some explanations.
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-2925:
------------------------
Attachment: HIVE-2925.2.patch.txt
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.1.patch.txt, HIVE-2925.2.patch.txt, HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2925:
------------------------------
Attachment: HIVE-2925.D2607.2.patch
navis updated the revision "HIVE-2925 [jira] Support non-MR fetching for simple queries with select/limit/filter operations only".
Reviewers: JIRA
1. Fixed handling of predicates pushed-down to TS
2. Added configuration for disabling aggresive fetch task conversion
3. Fixed duplicated adding Partitions to ReadEntity set
Seemed not yet complete.
REVISION DETAIL
https://reviews.facebook.net/D2607
AFFECTED FILES
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
ql/src/java/org/apache/hadoop/hive/ql/Driver.java
ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/FetchTask.java
ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java
ql/src/java/org/apache/hadoop/hive/ql/parse/QB.java
ql/src/java/org/apache/hadoop/hive/ql/parse/QBParseInfo.java
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
ql/src/java/org/apache/hadoop/hive/ql/plan/FetchWork.java
ql/src/test/queries/clientpositive/nonmr_fetch.q
ql/src/test/results/clientpositive/nonmr_fetch.q.out
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-2925:
-----------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed. Thanks Navis
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.1.patch.txt, HIVE-2925.2.patch.txt, HIVE-2925.3.patch.txt, HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-2925:
-----------------------------
Status: Open (was: Patch Available)
comments on phabricator
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Anoop Sam John (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13261588#comment-13261588 ]
Anoop Sam John commented on HIVE-2925:
--------------------------------------
Currently when we use the Hbase integration of Hive, querying data from HBase through Hive also will create MR job right? I think this feature can be useful there also. Ideally this might not need MR.
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-2925:
------------------------
Status: Patch Available (was: Open)
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.1.patch.txt, HIVE-2925.2.patch.txt, HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13423754#comment-13423754 ]
Namit Jain commented on HIVE-2925:
----------------------------------
Can you break it into 2 separate jiras --
reduce inputs for limits should be an optimization step
and
Support non-MR fetching for simple queries with select/limit/filter operations only
I think, moving the limit stuff to an optimizer step is an excellent idea, but it might be easier to review/checkin independently
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-2925:
-----------------------------
Status: Patch Available (was: Open)
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-2925:
------------------------
Affects Version/s: 0.10.0
Status: Patch Available (was: Open)
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-2925:
------------------------
Status: Patch Available (was: Open)
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13423743#comment-13423743 ]
Namit Jain commented on HIVE-2925:
----------------------------------
Can you refresh ?
The patch is not applying cleanly anymore.
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13423792#comment-13423792 ]
Namit Jain commented on HIVE-2925:
----------------------------------
It would be very difficult to deploy it this way.
In general, I can think of the following:
1. For queries with limits, this optimization should be enabled.
2. Ideally, it would be good, if there is a threshold of the limit.
3. For queries without limits, given the fact that we dont a cost based optimizer, it might be a good to
have a threshold on the total input data.
I mean, in general, non MR fetch might make sense for the following.
1. select from a small table (where small is configurable)
2. select from a big table is OK if there is a limit
Note that, it is still possible to get a plan where this optimization might make not sense.
For eg: select col1 from T where col2 = 10 limit 10;
It is possible that there are very rows for which col2 is 10, so not having a MR job may really slow down this query.
Solving that would be more difficult without more statistics.
But, it may be a good idea to add more config parameters to tune the hive.aggresive.fetch.task.conversion appropriately.
It can also be done in a follow-up patch, and is independent of this.
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-2925:
------------------------
Status: Patch Available (was: Open)
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2925:
------------------------------
Attachment: HIVE-2925.D2607.3.patch
navis updated the revision "HIVE-2925 [jira] Support non-MR fetching for simple queries with select/limit/filter operations only".
Reviewers: JIRA
1. Fixed test results (passed all tests)
REVISION DETAIL
https://reviews.facebook.net/D2607
AFFECTED FILES
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
ql/src/java/org/apache/hadoop/hive/ql/Driver.java
ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/FetchTask.java
ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java
ql/src/java/org/apache/hadoop/hive/ql/parse/QB.java
ql/src/java/org/apache/hadoop/hive/ql/parse/QBParseInfo.java
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
ql/src/java/org/apache/hadoop/hive/ql/plan/FetchWork.java
ql/src/test/queries/clientpositive/nonmr_fetch.q
ql/src/test/results/clientpositive/nonmr_fetch.q.out
To: JIRA, navis
Cc: njain
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427162#comment-13427162 ]
Namit Jain commented on HIVE-2925:
----------------------------------
+1
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-2925:
------------------------
Status: Patch Available (was: Open)
Passed all tests
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.1.patch.txt, HIVE-2925.2.patch.txt, HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13432951#comment-13432951 ]
Namit Jain commented on HIVE-2925:
----------------------------------
[~navis], can you refresh ?
Since, jira was down for some time, I could not take a look.
Please refresh, I will try to get it in asap
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.1.patch.txt, HIVE-2925.2.patch.txt, HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13246467#comment-13246467 ]
Phabricator commented on HIVE-2925:
-----------------------------------
njain has commented on the revision "HIVE-2925 [jira] Support non-MR fetching for simple queries with select/limit/filter operations only".
I will review this soon in detail.
But, one comment I had is that this is a backward incompatible change.
Can you add a new conf. parameter to control this ?
The default behavior of this parameter can be true, but we definitely need to provide a
knob to the end user to turn this off.
REVISION DETAIL
https://reviews.facebook.net/D2607
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437216#comment-13437216 ]
Hudson commented on HIVE-2925:
------------------------------
Integrated in Hive-trunk-h0.21 #1615 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1615/])
HIVE-2925 - Support non-MR fetching for simple queries with select/limit/filter operations only (Navis via namit) (Revision 1374389)
Result = FAILURE
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1374389
Files :
* /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
* /hive/trunk/conf/hive-default.xml.template
* /hive/trunk/hbase-handler/src/test/results/positive/hbase_ppd_key_range.q.out
* /hive/trunk/hbase-handler/src/test/results/positive/hbase_pushdown.q.out
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecMapperContext.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FetchTask.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ListSinkOperator.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/IOContext.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SimpleFetchOptimizer.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/GlobalLimitCtx.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/PrunedPartitionList.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/QB.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/QBParseInfo.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/FetchWork.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ListSinkDesc.java
* /hive/trunk/ql/src/test/queries/clientpositive/nonmr_fetch.q
* /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat2.q
* /hive/trunk/ql/src/test/results/clientpositive/input.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input0.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input_limit.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input_part0.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input_part3.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input_part4.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input_part8.q.out
* /hive/trunk/ql/src/test/results/clientpositive/nonmr_fetch.q.out
* /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppr_pushdown3.q.out
* /hive/trunk/ql/src/test/results/clientpositive/regex_col.q.out
* /hive/trunk/ql/src/test/results/clientpositive/source.q.out
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/DelegatedListObjectInspector.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/DelegatedMapObjectInspector.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/DelegatedObjectInspectorFactory.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/DelegatedStructObjectInspector.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/DelegatedUnionObjectInspector.java
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.1.patch.txt, HIVE-2925.2.patch.txt, HIVE-2925.3.patch.txt, HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13436589#comment-13436589 ]
Namit Jain commented on HIVE-2925:
----------------------------------
looks good - will start testing
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.1.patch.txt, HIVE-2925.2.patch.txt, HIVE-2925.3.patch.txt, HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2925:
------------------------------
Attachment: HIVE-2925.D2607.1.patch
navis requested code review of "HIVE-2925 [jira] Support non-MR fetching for simple queries with select/limit/filter operations only".
Reviewers: JIRA
DPAL-1039 Support non-MR fetching for simple queries with select/limit/filter operations only
It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
For that kind of simple queries, using fetch task would make them happy.
TEST PLAN
EMPTY
REVISION DETAIL
https://reviews.facebook.net/D2607
AFFECTED FILES
ql/src/java/org/apache/hadoop/hive/ql/Driver.java
ql/src/java/org/apache/hadoop/hive/ql/exec/FetchTask.java
ql/src/java/org/apache/hadoop/hive/ql/parse/QB.java
ql/src/java/org/apache/hadoop/hive/ql/parse/QBParseInfo.java
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
ql/src/java/org/apache/hadoop/hive/ql/plan/FetchWork.java
ql/src/test/queries/clientpositive/nonmr_fetch.q
ql/src/test/results/clientpositive/nonmr_fetch.q.out
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/5943/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13423832#comment-13423832 ]
Namit Jain commented on HIVE-2925:
----------------------------------
I was thinking about it more - what I said above cannot be performed in a custom hook.
In a pre-semantic hook, you dont have enough information --- the type of the query, stats etc.
In a post-semantic hook, it is too late.
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13424418#comment-13424418 ]
Namit Jain commented on HIVE-2925:
----------------------------------
Copying a mail I sent to dev@ for review....
Currently, hive does not launch map-reduce jobs for the following queries:
select * from <T> where <condition on partition columns> (limit <n>)?
This behavior is not configurable, and cannot be altered.
HIVE-2925 wants to extend this behavior. The goal is not to spawn map-reduce jobs for the following queries:
Select <expr> from <T> where <any condition> (limit <n>)?
It is currently controlled by one parameter: hive.aggressive.fetch.task.conversion, based on which it is decided, whether to spawn
map-reduce jobs or not for the queries of the above type. Note that this can be beneficial for certain types of queries, since it is
avoiding the expensive step of spawning map-reduce. However, it can be pretty expensive for certain types of queries: selecting
a very large number of rows, the query having a very selective filter (which is satisfied by a very number of rows, and therefore involves
scanning a very large table) etc. The user does not have any control on this. Note that it cannot be done by hooks, since the pre-semantic
hooks does not have enough information: type of the query, inputs etc. and it is too late to do anything in the post-semantic hook (the
query plan has already been altered).
I would like to propose the following configuration parameters to control this behavior.
hive.fetch.task.conversion: true, false, auto
If the value is true, then all queries with only selects and filters will be converted
If the value is false, then no query will be converted
If the value is auto (which should be the default behavior), there should be additional parameters to control the semantics.
hive.fetch.task.auto.limit.threshold ---> integer value X1
hive.fetch.task.auto.inputsize.threshold ---> integer value X2
If either the query has a limit lower than X1, or the input size is smaller than X2, the queries containing only filters and selects will be converted to not use
map-reudce jobs.
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13428570#comment-13428570 ]
Namit Jain commented on HIVE-2925:
----------------------------------
One test: partition_wise_fileformat2.q still fails.
Can you debug ?
It has a select * query with different file formats for different partitions.
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.1.patch.txt, HIVE-2925.2.patch.txt, HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-2925:
-----------------------------
Status: Open (was: Patch Available)
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427153#comment-13427153 ]
Namit Jain commented on HIVE-2925:
----------------------------------
question ?
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427971#comment-13427971 ]
Namit Jain commented on HIVE-2925:
----------------------------------
comments
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.1.patch.txt, HIVE-2925.2.patch.txt, HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-2925:
-----------------------------
Status: Open (was: Patch Available)
Some tests failed -
Some failures in TestHBaseCliDriver
TesCliDriver (bucketcontext_2.q, bucketcontext_3.q,,,,)
Can you take a look ?
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.1.patch.txt, HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-2925:
-----------------------------
Status: Open (was: Patch Available)
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.1.patch.txt, HIVE-2925.2.patch.txt, HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-2925:
-----------------------------
Status: Open (was: Patch Available)
minor comments on phabricator
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2925:
------------------------------
Attachment: HIVE-2925.D2607.4.patch
navis updated the revision "HIVE-2925 [jira] Support non-MR fetching for simple queries with select/limit/filter operations only".
Reviewers: JIRA
Rebased to trunk.. Phabricator confuses me in theses days.
REVISION DETAIL
https://reviews.facebook.net/D2607
AFFECTED FILES
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
ql/src/java/org/apache/hadoop/hive/ql/Driver.java
ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/FetchTask.java
ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java
ql/src/java/org/apache/hadoop/hive/ql/parse/QB.java
ql/src/java/org/apache/hadoop/hive/ql/parse/QBParseInfo.java
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
ql/src/java/org/apache/hadoop/hive/ql/plan/FetchWork.java
ql/src/test/queries/clientpositive/nonmr_fetch.q
ql/src/test/results/clientpositive/nonmr_fetch.q.out
To: JIRA, navis
Cc: njain
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-2925:
-----------------------------
Status: Open (was: Patch Available)
test changes
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-2925:
-----------------------------
Status: Open (was: Patch Available)
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.1.patch.txt, HIVE-2925.2.patch.txt, HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2925) Support non-MR fetching for simple
queries with select/limit/filter operations only
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-2925:
------------------------
Status: Patch Available (was: Open)
> Support non-MR fetching for simple queries with select/limit/filter operations only
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2925
> URL: https://issues.apache.org/jira/browse/HIVE-2925
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2925.D2607.1.patch, HIVE-2925.D2607.2.patch, HIVE-2925.D2607.3.patch, HIVE-2925.D2607.4.patch
>
>
> It's trivial but frequently asked by end-users. Currently, select queries with simple conditions or limit should run MR job which takes some time especially for big tables, making the people irritated.
> For that kind of simple queries, using fetch task would make them happy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira