You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Philip Zeyliger (JIRA)" <ji...@apache.org> on 2010/10/06 00:35:32 UTC

[jira] Created: (HIVE-1692) FetchOperator.getInputFormatFromCache hides causal exception

FetchOperator.getInputFormatFromCache hides causal exception
------------------------------------------------------------

                 Key: HIVE-1692
                 URL: https://issues.apache.org/jira/browse/HIVE-1692
             Project: Hadoop Hive
          Issue Type: Improvement
          Components: Query Processor
    Affects Versions: 0.7.0
            Reporter: Philip Zeyliger
            Priority: Minor
             Fix For: 0.7.0


There's a line in FetchOperator.getInputFormatFromCache that catches all exceptions and re-throws IOException instead, hiding the original cause.  I ran into this, naturally, and wish to fix it.  Patch below is trivial.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HIVE-1692) FetchOperator.getInputFormatFromCache hides causal exception

Posted by "Philip Zeyliger (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HIVE-1692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Philip Zeyliger updated HIVE-1692:
----------------------------------

    Attachment: HIVE-1692.patch.txt

I'll spare folks downloading the attachment:

{noformat}
-            + inputFormatClass.getName() + " as specified in mapredWork!");
+            + inputFormatClass.getName() + " as specified in mapredWork!", e);
{noformat}

> FetchOperator.getInputFormatFromCache hides causal exception
> ------------------------------------------------------------
>
>                 Key: HIVE-1692
>                 URL: https://issues.apache.org/jira/browse/HIVE-1692
>             Project: Hadoop Hive
>          Issue Type: Improvement
>          Components: Query Processor
>    Affects Versions: 0.7.0
>            Reporter: Philip Zeyliger
>            Priority: Minor
>             Fix For: 0.7.0
>
>         Attachments: HIVE-1692.patch.txt
>
>
> There's a line in FetchOperator.getInputFormatFromCache that catches all exceptions and re-throws IOException instead, hiding the original cause.  I ran into this, naturally, and wish to fix it.  Patch below is trivial.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HIVE-1692) FetchOperator.getInputFormatFromCache hides causal exception

Posted by "Philip Zeyliger (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HIVE-1692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12918422#action_12918422 ] 

Philip Zeyliger commented on HIVE-1692:
---------------------------------------

BTW, to illustrate what a difference 3 characters make, compare debugging the following two errors:

(no patch)
{noformat}
10/10/05 21:55:39 ERROR CliDriver: Failed with exception java.io.IOException:java.io.IOException: Cannot create an instance of InputFormat class org.apache.hadoop.mapred.TextInputFormat as specified in mapredWork!
java.io.IOException: java.io.IOException: Cannot create an instance of InputFormat class org.apache.hadoop.mapred.TextInputFormat as specified in mapredWork!
	at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:271)
	at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:113)
	at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:657)
	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:131)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
Caused by: java.io.IOException: Cannot create an instance of InputFormat class org.apache.hadoop.mapred.TextInputFormat as specified in mapredWork!
	at org.apache.hadoop.hive.ql.exec.FetchOperator.getInputFormatFromCache(FetchOperator.java:113)
	at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:214)
	at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:250)
	... 10 more
{noformat}

(patch)
{noformat}

10/10/05 21:54:03 ERROR CliDriver: Failed with exception java.io.IOException:java.io.IOException: Cannot create an instance of InputFormat class org.apache.hadoop.mapred.TextInputFormat as specified in mapredWork!
java.io.IOException: java.io.IOException: Cannot create an instance of InputFormat class org.apache.hadoop.mapred.TextInputFormat as specified in mapredWork!
	at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:271)
	at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:113)
	at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:657)
	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:131)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
Caused by: java.io.IOException: Cannot create an instance of InputFormat class org.apache.hadoop.mapred.TextInputFormat as specified in mapredWork!
	at org.apache.hadoop.hive.ql.exec.FetchOperator.getInputFormatFromCache(FetchOperator.java:113)
	at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:214)
	at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:250)
	... 10 more
Caused by: java.lang.RuntimeException: Error in configuring object
	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
	at org.apache.hadoop.hive.ql.exec.FetchOperator.getInputFormatFromCache(FetchOperator.java:109)
	... 12 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
	... 15 more
Caused by: java.lang.IllegalArgumentException: Compression codec com.hadoop.compression.lzo.LzoCodec not found.
	at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:96)
	at org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:134)
	at org.apache.hadoop.mapred.TextInputFormat.configure(TextInputFormat.java:41)
	... 20 more
Caused by: java.lang.ClassNotFoundException: com.hadoop.compression.lzo.LzoCodec
	at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:247)
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:907)
	at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:89)
	... 22 more
{noformat}

Obviously, the latter is better, because it gets to the root cause of the problem.  Searching the user list turned out that I wasn't the first to run into this.

> FetchOperator.getInputFormatFromCache hides causal exception
> ------------------------------------------------------------
>
>                 Key: HIVE-1692
>                 URL: https://issues.apache.org/jira/browse/HIVE-1692
>             Project: Hadoop Hive
>          Issue Type: Improvement
>          Components: Query Processor
>    Affects Versions: 0.7.0
>            Reporter: Philip Zeyliger
>            Priority: Minor
>             Fix For: 0.7.0
>
>         Attachments: HIVE-1692.patch.txt
>
>
> There's a line in FetchOperator.getInputFormatFromCache that catches all exceptions and re-throws IOException instead, hiding the original cause.  I ran into this, naturally, and wish to fix it.  Patch below is trivial.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.