You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Thiruvel Thirumoolan (Created) (JIRA)" <ji...@apache.org> on 2012/03/30 11:38:35 UTC
[jira] [Created] (HIVE-2913) BlockMergeTask Doesn't Honor Job
Configuration Properties when used directly
BlockMergeTask Doesn't Honor Job Configuration Properties when used directly
----------------------------------------------------------------------------
Key: HIVE-2913
URL: https://issues.apache.org/jira/browse/HIVE-2913
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.8.1
Reporter: Thiruvel Thirumoolan
Assignee: Thiruvel Thirumoolan
Priority: Minor
Fix For: 0.9.0
BlockMergeTask has a main() and when used directly (instead of say partition concatenate feature), the -jobconf arguments are not honored. This is not something most people directly use.
Usage:
BlockMergeTask -input <colon seperated input paths> -outputDir outputDir [-jobconf k1=v1 [-jobconf k2=v2] ...]
To reproduce:
Run BlockMergeTask with say -jobconf mapred.job.name=test and launched job will have a different name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2913) BlockMergeTask Doesn't Honor Job
Configuration Properties when used directly
Posted by "Thiruvel Thirumoolan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thiruvel Thirumoolan updated HIVE-2913:
---------------------------------------
Attachment: HIVE-2913_1.patch
> BlockMergeTask Doesn't Honor Job Configuration Properties when used directly
> ----------------------------------------------------------------------------
>
> Key: HIVE-2913
> URL: https://issues.apache.org/jira/browse/HIVE-2913
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.8.1
> Reporter: Thiruvel Thirumoolan
> Assignee: Thiruvel Thirumoolan
> Priority: Minor
> Fix For: 0.9.0
>
> Attachments: HIVE-2913_1.patch
>
>
> BlockMergeTask has a main() and when used directly (instead of say partition concatenate feature), the -jobconf arguments are not honored. This is not something most people directly use.
> Usage:
> BlockMergeTask -input <colon seperated input paths> -outputDir outputDir [-jobconf k1=v1 [-jobconf k2=v2] ...]
> To reproduce:
> Run BlockMergeTask with say -jobconf mapred.job.name=test and launched job will have a different name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2913) BlockMergeTask Doesn't Honor Job
Configuration Properties when used directly
Posted by "Amareshwari Sriramadasu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13244971#comment-13244971 ]
Amareshwari Sriramadasu commented on HIVE-2913:
-----------------------------------------------
+1. Running tests. Will commit once tests pass.
> BlockMergeTask Doesn't Honor Job Configuration Properties when used directly
> ----------------------------------------------------------------------------
>
> Key: HIVE-2913
> URL: https://issues.apache.org/jira/browse/HIVE-2913
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.8.1
> Reporter: Thiruvel Thirumoolan
> Assignee: Thiruvel Thirumoolan
> Priority: Minor
> Fix For: 0.9.0
>
> Attachments: HIVE-2913.D2547.1.patch, HIVE-2913_1.patch, blockmerge.sh
>
>
> BlockMergeTask has a main() and when used directly (instead of say partition concatenate feature), the -jobconf arguments are not honored. This is not something most people directly use.
> Usage:
> BlockMergeTask -input <colon seperated input paths> -outputDir outputDir [-jobconf k1=v1 [-jobconf k2=v2] ...]
> To reproduce:
> Run BlockMergeTask with say -jobconf mapred.job.name=test and launched job will have a different name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2913) BlockMergeTask Doesn't Honor Job
Configuration Properties when used directly
Posted by "Thiruvel Thirumoolan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thiruvel Thirumoolan updated HIVE-2913:
---------------------------------------
Attachment: blockmerge.sh
Simple shell script that helped to test. Place it under $HIVE_HOME/bin/ext.
> BlockMergeTask Doesn't Honor Job Configuration Properties when used directly
> ----------------------------------------------------------------------------
>
> Key: HIVE-2913
> URL: https://issues.apache.org/jira/browse/HIVE-2913
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.8.1
> Reporter: Thiruvel Thirumoolan
> Assignee: Thiruvel Thirumoolan
> Priority: Minor
> Fix For: 0.9.0
>
> Attachments: HIVE-2913_1.patch, blockmerge.sh
>
>
> BlockMergeTask has a main() and when used directly (instead of say partition concatenate feature), the -jobconf arguments are not honored. This is not something most people directly use.
> Usage:
> BlockMergeTask -input <colon seperated input paths> -outputDir outputDir [-jobconf k1=v1 [-jobconf k2=v2] ...]
> To reproduce:
> Run BlockMergeTask with say -jobconf mapred.job.name=test and launched job will have a different name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2913) BlockMergeTask Doesn't Honor Job
Configuration Properties when used directly
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13244973#comment-13244973 ]
Phabricator commented on HIVE-2913:
-----------------------------------
amareshwarisr has accepted the revision "HIVE-2913 [jira] BlockMergeTask Doesn't Honor Job Configuration Properties when used directly".
+1
REVISION DETAIL
https://reviews.facebook.net/D2547
BRANCH
svn
> BlockMergeTask Doesn't Honor Job Configuration Properties when used directly
> ----------------------------------------------------------------------------
>
> Key: HIVE-2913
> URL: https://issues.apache.org/jira/browse/HIVE-2913
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.8.1
> Reporter: Thiruvel Thirumoolan
> Assignee: Thiruvel Thirumoolan
> Priority: Minor
> Fix For: 0.9.0
>
> Attachments: HIVE-2913.D2547.1.patch, HIVE-2913_1.patch, blockmerge.sh
>
>
> BlockMergeTask has a main() and when used directly (instead of say partition concatenate feature), the -jobconf arguments are not honored. This is not something most people directly use.
> Usage:
> BlockMergeTask -input <colon seperated input paths> -outputDir outputDir [-jobconf k1=v1 [-jobconf k2=v2] ...]
> To reproduce:
> Run BlockMergeTask with say -jobconf mapred.job.name=test and launched job will have a different name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2913) BlockMergeTask Doesn't Honor Job
Configuration Properties when used directly
Posted by "Thiruvel Thirumoolan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thiruvel Thirumoolan updated HIVE-2913:
---------------------------------------
Status: Patch Available (was: Open)
> BlockMergeTask Doesn't Honor Job Configuration Properties when used directly
> ----------------------------------------------------------------------------
>
> Key: HIVE-2913
> URL: https://issues.apache.org/jira/browse/HIVE-2913
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.8.1
> Reporter: Thiruvel Thirumoolan
> Assignee: Thiruvel Thirumoolan
> Priority: Minor
> Fix For: 0.9.0
>
> Attachments: HIVE-2913.D2547.1.patch, HIVE-2913_1.patch, blockmerge.sh
>
>
> BlockMergeTask has a main() and when used directly (instead of say partition concatenate feature), the -jobconf arguments are not honored. This is not something most people directly use.
> Usage:
> BlockMergeTask -input <colon seperated input paths> -outputDir outputDir [-jobconf k1=v1 [-jobconf k2=v2] ...]
> To reproduce:
> Run BlockMergeTask with say -jobconf mapred.job.name=test and launched job will have a different name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2913) BlockMergeTask Doesn't Honor Job
Configuration Properties when used directly
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13245592#comment-13245592 ]
Hudson commented on HIVE-2913:
------------------------------
Integrated in Hive-trunk-h0.21 #1350 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1350/])
HIVE-2913. Fixes BlockMergeTask to honor Configuration Properties when used directly. (Thiruvel Thirumoolan via amareshwari) (Revision 1308812)
Result = FAILURE
amareshwari : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1308812
Files :
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/BlockMergeTask.java
> BlockMergeTask Doesn't Honor Job Configuration Properties when used directly
> ----------------------------------------------------------------------------
>
> Key: HIVE-2913
> URL: https://issues.apache.org/jira/browse/HIVE-2913
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.8.1
> Reporter: Thiruvel Thirumoolan
> Assignee: Thiruvel Thirumoolan
> Priority: Minor
> Fix For: 0.9.0
>
> Attachments: HIVE-2913.D2547.1.patch, HIVE-2913_1.patch, blockmerge.sh
>
>
> BlockMergeTask has a main() and when used directly (instead of say partition concatenate feature), the -jobconf arguments are not honored. This is not something most people directly use.
> Usage:
> BlockMergeTask -input <colon seperated input paths> -outputDir outputDir [-jobconf k1=v1 [-jobconf k2=v2] ...]
> To reproduce:
> Run BlockMergeTask with say -jobconf mapred.job.name=test and launched job will have a different name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2913) BlockMergeTask Doesn't Honor Job
Configuration Properties when used directly
Posted by "Amareshwari Sriramadasu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HIVE-2913:
------------------------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
I just committed this.
Thanks Thiruvel!
> BlockMergeTask Doesn't Honor Job Configuration Properties when used directly
> ----------------------------------------------------------------------------
>
> Key: HIVE-2913
> URL: https://issues.apache.org/jira/browse/HIVE-2913
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.8.1
> Reporter: Thiruvel Thirumoolan
> Assignee: Thiruvel Thirumoolan
> Priority: Minor
> Fix For: 0.9.0
>
> Attachments: HIVE-2913.D2547.1.patch, HIVE-2913_1.patch, blockmerge.sh
>
>
> BlockMergeTask has a main() and when used directly (instead of say partition concatenate feature), the -jobconf arguments are not honored. This is not something most people directly use.
> Usage:
> BlockMergeTask -input <colon seperated input paths> -outputDir outputDir [-jobconf k1=v1 [-jobconf k2=v2] ...]
> To reproduce:
> Run BlockMergeTask with say -jobconf mapred.job.name=test and launched job will have a different name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2913) BlockMergeTask Doesn't Honor Job
Configuration Properties when used directly
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2913:
------------------------------
Attachment: HIVE-2913.D2547.1.patch
thiruvel requested code review of "HIVE-2913 [jira] BlockMergeTask Doesn't Honor Job Configuration Properties when used directly".
Reviewers: JIRA
https://issues.apache.org/jira/browse/HIVE-2913
HIVE-2913: Running BlockMergeTask through its main() interface should honor jobconf properties
BlockMergeTask has a main() and when used directly (instead of say partition concatenate feature), the -jobconf arguments are not honored. This is not something most people directly use.
Usage:
BlockMergeTask -input <colon seperated input paths> -outputDir outputDir [-jobconf k1=v1 [-jobconf k2=v2] ...]
To reproduce:
Run BlockMergeTask with say -jobconf mapred.job.name=test and launched job will have a different name.
TEST PLAN
EMPTY
REVISION DETAIL
https://reviews.facebook.net/D2547
AFFECTED FILES
ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/BlockMergeTask.java
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/5793/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
> BlockMergeTask Doesn't Honor Job Configuration Properties when used directly
> ----------------------------------------------------------------------------
>
> Key: HIVE-2913
> URL: https://issues.apache.org/jira/browse/HIVE-2913
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.8.1
> Reporter: Thiruvel Thirumoolan
> Assignee: Thiruvel Thirumoolan
> Priority: Minor
> Fix For: 0.9.0
>
> Attachments: HIVE-2913.D2547.1.patch, HIVE-2913_1.patch, blockmerge.sh
>
>
> BlockMergeTask has a main() and when used directly (instead of say partition concatenate feature), the -jobconf arguments are not honored. This is not something most people directly use.
> Usage:
> BlockMergeTask -input <colon seperated input paths> -outputDir outputDir [-jobconf k1=v1 [-jobconf k2=v2] ...]
> To reproduce:
> Run BlockMergeTask with say -jobconf mapred.job.name=test and launched job will have a different name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira