You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org> on 2010/03/05 23:35:27 UTC
[jira] Commented: (HADOOP-6588) CompressionCodecFactory throws
IllegalArgumentException
[ https://issues.apache.org/jira/browse/HADOOP-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12842072#action_12842072 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-6588:
------------------------------------------------
> That is because you specified LzoCodec class in your conf file. ...
It probably is the case.
> ... I think it is better to leave the behavior as-is instead of issuing log messages.
I am fine for keeping the current behavior, i.e. throwing an exception. However, it is better to change the exception since it is not an IllegalArgumentException.
Also, CompressionCodecFactory constructor calls getCodecClasses(..) which ultimately calls Class.forName(..). As a consequence, Class.forName(..) will be invoked for every TextInputFormat.isSplitable(..). It is unnecessary and expensive.
> CompressionCodecFactory throws IllegalArgumentException
> -------------------------------------------------------
>
> Key: HADOOP-6588
> URL: https://issues.apache.org/jira/browse/HADOOP-6588
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Reporter: Tsz Wo (Nicholas), SZE
> Attachments: c6588_20100222.patch
>
>
> WordCount does not run. :(
> {noformat}
> java.lang.IllegalArgumentException: Compression codec com.hadoop.compression.lzo.LzoCodec not found.
> at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:96)
> at org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:134)
> at org.apache.hadoop.mapreduce.lib.input.TextInputFormat.isSplitable(TextInputFormat.java:46)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:247)
> at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:886)
> at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:780)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:444)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:459)
> at org.apache.hadoop.examples.WordCount.main(WordCount.java:67)
> ...
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.