You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Navis (JIRA)" <ji...@apache.org> on 2012/06/30 05:39:42 UTC
[jira] [Created] (HIVE-3219) BucketizedHiveInputFormat should be
used only with SMBJoin
Navis created HIVE-3219:
---------------------------
Summary: BucketizedHiveInputFormat should be used only with SMBJoin
Key: HIVE-3219
URL: https://issues.apache.org/jira/browse/HIVE-3219
Project: Hive
Issue Type: Improvement
Components: Query Processor
Affects Versions: 0.10.0
Reporter: Navis
Assignee: Navis
Priority: Trivial
For SMBJoin, BucketizedHiveInputFormat should(could?) be used with set command.
{noformat}
set hive.input.format = org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;
{noformat}
But this affects all of the MR taks in a query, which might not be intended.
With some configuration, it should be automatically configured only for MR tasks which contains SMBJoin.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3219) BucketizedHiveInputFormat should be
automatically used with SMBJoin
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-3219:
-----------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed. Thanks Navis
> BucketizedHiveInputFormat should be automatically used with SMBJoin
> -------------------------------------------------------------------
>
> Key: HIVE-3219
> URL: https://issues.apache.org/jira/browse/HIVE-3219
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3219.1.patch.txt
>
>
> For SMBJoin, BucketizedHiveInputFormat should(could?) be used with set command.
> {noformat}
> set hive.input.format = org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;
> {noformat}
> But this affects all of the MR taks in a query, which might not be intended.
> With some configuration, it should be automatically configured only for MR tasks which contains SMBJoin.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3219) BucketizedHiveInputFormat should be
automatically used with SMBJoin
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-3219:
------------------------
Status: Patch Available (was: Open)
> BucketizedHiveInputFormat should be automatically used with SMBJoin
> -------------------------------------------------------------------
>
> Key: HIVE-3219
> URL: https://issues.apache.org/jira/browse/HIVE-3219
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3219.1.patch.txt
>
>
> For SMBJoin, BucketizedHiveInputFormat should(could?) be used with set command.
> {noformat}
> set hive.input.format = org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;
> {noformat}
> But this affects all of the MR taks in a query, which might not be intended.
> With some configuration, it should be automatically configured only for MR tasks which contains SMBJoin.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3219) BucketizedHiveInputFormat should be
automatically used with SMBJoin
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-3219:
-----------------------------
Status: Open (was: Patch Available)
minor comments on the review
> BucketizedHiveInputFormat should be automatically used with SMBJoin
> -------------------------------------------------------------------
>
> Key: HIVE-3219
> URL: https://issues.apache.org/jira/browse/HIVE-3219
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3219.1.patch.txt
>
>
> For SMBJoin, BucketizedHiveInputFormat should(could?) be used with set command.
> {noformat}
> set hive.input.format = org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;
> {noformat}
> But this affects all of the MR taks in a query, which might not be intended.
> With some configuration, it should be automatically configured only for MR tasks which contains SMBJoin.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3219) BucketizedHiveInputFormat should be
automatically used with SMBJoin
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13416110#comment-13416110 ]
Navis commented on HIVE-3219:
-----------------------------
Isn't it possible to someone want to override BucketizedHiveInputFormat?
> BucketizedHiveInputFormat should be automatically used with SMBJoin
> -------------------------------------------------------------------
>
> Key: HIVE-3219
> URL: https://issues.apache.org/jira/browse/HIVE-3219
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3219.1.patch.txt
>
>
> For SMBJoin, BucketizedHiveInputFormat should(could?) be used with set command.
> {noformat}
> set hive.input.format = org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;
> {noformat}
> But this affects all of the MR taks in a query, which might not be intended.
> With some configuration, it should be automatically configured only for MR tasks which contains SMBJoin.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3219) BucketizedHiveInputFormat should be
used only with SMBJoin
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-3219:
------------------------
Attachment: HIVE-3219.1.patch.txt
> BucketizedHiveInputFormat should be used only with SMBJoin
> ----------------------------------------------------------
>
> Key: HIVE-3219
> URL: https://issues.apache.org/jira/browse/HIVE-3219
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3219.1.patch.txt
>
>
> For SMBJoin, BucketizedHiveInputFormat should(could?) be used with set command.
> {noformat}
> set hive.input.format = org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;
> {noformat}
> But this affects all of the MR taks in a query, which might not be intended.
> With some configuration, it should be automatically configured only for MR tasks which contains SMBJoin.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3219) BucketizedHiveInputFormat should be
automatically used with SMBJoin
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13426366#comment-13426366 ]
Namit Jain commented on HIVE-3219:
----------------------------------
+1
> BucketizedHiveInputFormat should be automatically used with SMBJoin
> -------------------------------------------------------------------
>
> Key: HIVE-3219
> URL: https://issues.apache.org/jira/browse/HIVE-3219
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3219.1.patch.txt
>
>
> For SMBJoin, BucketizedHiveInputFormat should(could?) be used with set command.
> {noformat}
> set hive.input.format = org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;
> {noformat}
> But this affects all of the MR taks in a query, which might not be intended.
> With some configuration, it should be automatically configured only for MR tasks which contains SMBJoin.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3219) BucketizedHiveInputFormat should be
used only with SMBJoin
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-3219:
------------------------
Status: Patch Available (was: Open)
https://reviews.facebook.net/D3939
> BucketizedHiveInputFormat should be used only with SMBJoin
> ----------------------------------------------------------
>
> Key: HIVE-3219
> URL: https://issues.apache.org/jira/browse/HIVE-3219
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
>
> For SMBJoin, BucketizedHiveInputFormat should(could?) be used with set command.
> {noformat}
> set hive.input.format = org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;
> {noformat}
> But this affects all of the MR taks in a query, which might not be intended.
> With some configuration, it should be automatically configured only for MR tasks which contains SMBJoin.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3219) BucketizedHiveInputFormat should be
automatically used with SMBJoin
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-3219:
-----------------------------
Summary: BucketizedHiveInputFormat should be automatically used with SMBJoin (was: BucketizedHiveInputFormat should be used only with SMBJoin)
> BucketizedHiveInputFormat should be automatically used with SMBJoin
> -------------------------------------------------------------------
>
> Key: HIVE-3219
> URL: https://issues.apache.org/jira/browse/HIVE-3219
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3219.1.patch.txt
>
>
> For SMBJoin, BucketizedHiveInputFormat should(could?) be used with set command.
> {noformat}
> set hive.input.format = org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;
> {noformat}
> But this affects all of the MR taks in a query, which might not be intended.
> With some configuration, it should be automatically configured only for MR tasks which contains SMBJoin.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3219) BucketizedHiveInputFormat should be
automatically used with SMBJoin
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-3219:
-----------------------------
Status: Open (was: Patch Available)
> BucketizedHiveInputFormat should be automatically used with SMBJoin
> -------------------------------------------------------------------
>
> Key: HIVE-3219
> URL: https://issues.apache.org/jira/browse/HIVE-3219
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3219.1.patch.txt
>
>
> For SMBJoin, BucketizedHiveInputFormat should(could?) be used with set command.
> {noformat}
> set hive.input.format = org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;
> {noformat}
> But this affects all of the MR taks in a query, which might not be intended.
> With some configuration, it should be automatically configured only for MR tasks which contains SMBJoin.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3219) BucketizedHiveInputFormat should be
automatically used with SMBJoin
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-3219:
------------------------
Status: Patch Available (was: Open)
Addressed comments
> BucketizedHiveInputFormat should be automatically used with SMBJoin
> -------------------------------------------------------------------
>
> Key: HIVE-3219
> URL: https://issues.apache.org/jira/browse/HIVE-3219
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3219.1.patch.txt
>
>
> For SMBJoin, BucketizedHiveInputFormat should(could?) be used with set command.
> {noformat}
> set hive.input.format = org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;
> {noformat}
> But this affects all of the MR taks in a query, which might not be intended.
> With some configuration, it should be automatically configured only for MR tasks which contains SMBJoin.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3219) BucketizedHiveInputFormat should be
automatically used with SMBJoin
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13428239#comment-13428239 ]
Hudson commented on HIVE-3219:
------------------------------
Integrated in Hive-trunk-h0.21 #1589 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1589/])
HIVE-3219 BucketizedHiveInputFormat should be automatically used with SMBJoin
(Navis via namit) (Revision 1368872)
Result = FAILURE
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1368872
Files :
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/MapredWork.java
* /hive/trunk/ql/src/test/queries/clientpositive/bucketizedhiveinputformat_auto.q
* /hive/trunk/ql/src/test/results/clientpositive/bucketizedhiveinputformat_auto.q.out
> BucketizedHiveInputFormat should be automatically used with SMBJoin
> -------------------------------------------------------------------
>
> Key: HIVE-3219
> URL: https://issues.apache.org/jira/browse/HIVE-3219
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3219.1.patch.txt
>
>
> For SMBJoin, BucketizedHiveInputFormat should(could?) be used with set command.
> {noformat}
> set hive.input.format = org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;
> {noformat}
> But this affects all of the MR taks in a query, which might not be intended.
> With some configuration, it should be automatically configured only for MR tasks which contains SMBJoin.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira