You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Krishna Kumar (JIRA)" <ji...@apache.org> on 2011/08/26 07:35:29 UTC
[jira] [Created] (HIVE-2413) BlockMergeTask ignores
client-specified jars
BlockMergeTask ignores client-specified jars
--------------------------------------------
Key: HIVE-2413
URL: https://issues.apache.org/jira/browse/HIVE-2413
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Krishna Kumar
Priority: Minor
User-specified jars are not added to the hadoop tasks while executing a BlockMergeTask resulting in a ClassNotFoundException.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2413) BlockMergeTask ignores
client-specified jars
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13096481#comment-13096481 ]
Hudson commented on HIVE-2413:
------------------------------
Integrated in Hive-trunk-h0.21 #930 (See [https://builds.apache.org/job/Hive-trunk-h0.21/930/])
HIVE-2413: BlockMergeTask ignores client-specified jars (Krishna Kumar via He Yongqiang)
heyongqiang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1164656
Files :
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/BlockMergeTask.java
> BlockMergeTask ignores client-specified jars
> --------------------------------------------
>
> Key: HIVE-2413
> URL: https://issues.apache.org/jira/browse/HIVE-2413
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Krishna Kumar
> Assignee: Krishna Kumar
> Priority: Minor
> Attachments: HIVE-2413.v0.patch, HIVE-2413.v1.patch
>
>
> User-specified jars are not added to the hadoop tasks while executing a BlockMergeTask resulting in a ClassNotFoundException.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2413) BlockMergeTask ignores
client-specified jars
Posted by "He Yongqiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094278#comment-13094278 ]
He Yongqiang commented on HIVE-2413:
------------------------------------
+1, will commit after tests pass.
> BlockMergeTask ignores client-specified jars
> --------------------------------------------
>
> Key: HIVE-2413
> URL: https://issues.apache.org/jira/browse/HIVE-2413
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Krishna Kumar
> Assignee: Krishna Kumar
> Priority: Minor
> Attachments: HIVE-2413.v0.patch
>
>
> User-specified jars are not added to the hadoop tasks while executing a BlockMergeTask resulting in a ClassNotFoundException.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2413) BlockMergeTask ignores
client-specified jars
Posted by "Krishna Kumar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Krishna Kumar updated HIVE-2413:
--------------------------------
Attachment: HIVE-2413.v0.patch
Here is a simple fix which works for me, though there may be better ways of handling this (refactor mapredtask & blockmerge task to use a common routine / blockmerge as an mr operator).
No automated tests added.
> BlockMergeTask ignores client-specified jars
> --------------------------------------------
>
> Key: HIVE-2413
> URL: https://issues.apache.org/jira/browse/HIVE-2413
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Krishna Kumar
> Priority: Minor
> Attachments: HIVE-2413.v0.patch
>
>
> User-specified jars are not added to the hadoop tasks while executing a BlockMergeTask resulting in a ClassNotFoundException.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2413) BlockMergeTask ignores
client-specified jars
Posted by "He Yongqiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094962#comment-13094962 ]
He Yongqiang commented on HIVE-2413:
------------------------------------
[junit] java.lang.IllegalArgumentException: Can not create a Path from an empty string
[junit] at org.apache.hadoop.fs.Path.checkPathArg(Path.java:82)
[junit] at org.apache.hadoop.fs.Path.<init>(Path.java:90)
[junit] at org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient.java:602)
[junit] at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:761)
[junit] at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:730)
[junit] at org.apache.hadoop.hive.ql.io.rcfile.merge.BlockMergeTask.execute(BlockMergeTask.java:203)
[junit] at org.apache.hadoop.hive.ql.exec.DDLTask.mergeFiles(DDLTask.java:410)
[junit] at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:366)
[junit] at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:132)
[junit] at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
[junit] at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1343)
[junit] at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1134)
[junit] at org.apache.hadoop.hive.ql.Driver.run(Driver.java:943)
[junit] at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:253)
[junit] at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:210)
[junit] at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:401)
[junit] at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:336)
[junit] at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:638)
[junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_concatenate_indexed_table(TestCliDriver.java:1190)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
I got these error with a bunch of testcases. Here are some of them: rcfile_merge3.q, load_fs.q, alter_merge.q etc
can u take a look?
> BlockMergeTask ignores client-specified jars
> --------------------------------------------
>
> Key: HIVE-2413
> URL: https://issues.apache.org/jira/browse/HIVE-2413
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Krishna Kumar
> Assignee: Krishna Kumar
> Priority: Minor
> Attachments: HIVE-2413.v0.patch
>
>
> User-specified jars are not added to the hadoop tasks while executing a BlockMergeTask resulting in a ClassNotFoundException.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2413) BlockMergeTask ignores
client-specified jars
Posted by "Krishna Kumar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Krishna Kumar updated HIVE-2413:
--------------------------------
Assignee: Krishna Kumar
Status: Patch Available (was: Open)
> BlockMergeTask ignores client-specified jars
> --------------------------------------------
>
> Key: HIVE-2413
> URL: https://issues.apache.org/jira/browse/HIVE-2413
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Krishna Kumar
> Assignee: Krishna Kumar
> Priority: Minor
> Attachments: HIVE-2413.v0.patch
>
>
> User-specified jars are not added to the hadoop tasks while executing a BlockMergeTask resulting in a ClassNotFoundException.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2413) BlockMergeTask ignores
client-specified jars
Posted by "He Yongqiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
He Yongqiang updated HIVE-2413:
-------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
committed, thanks Krishna Kumar!
> BlockMergeTask ignores client-specified jars
> --------------------------------------------
>
> Key: HIVE-2413
> URL: https://issues.apache.org/jira/browse/HIVE-2413
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Krishna Kumar
> Assignee: Krishna Kumar
> Priority: Minor
> Attachments: HIVE-2413.v0.patch, HIVE-2413.v1.patch
>
>
> User-specified jars are not added to the hadoop tasks while executing a BlockMergeTask resulting in a ClassNotFoundException.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2413) BlockMergeTask ignores
client-specified jars
Posted by "Krishna Kumar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Krishna Kumar updated HIVE-2413:
--------------------------------
Attachment: HIVE-2413.v1.patch
Empty string not handled correctly in JC so handling it here...
> BlockMergeTask ignores client-specified jars
> --------------------------------------------
>
> Key: HIVE-2413
> URL: https://issues.apache.org/jira/browse/HIVE-2413
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Krishna Kumar
> Assignee: Krishna Kumar
> Priority: Minor
> Attachments: HIVE-2413.v0.patch, HIVE-2413.v1.patch
>
>
> User-specified jars are not added to the hadoop tasks while executing a BlockMergeTask resulting in a ClassNotFoundException.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira