You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Mark <st...@gmail.com> on 2011/10/31 18:49:05 UTC
Default Compression
I recently added the following to my core-site.xml
<property>
<name>io.compression.codecs</name>
<value>
org.apache.hadoop.io.compress.DefaultCodec,
org.apache.hadoop.io.compress.GzipCodec,
org.apache.hadoop.io.compress.BZip2Codec
</value>
</property>
However when I try and test a simple MR job I am seeing the following
errors in my log.
java.lang.IllegalArgumentException: Compression codec
org.apache.hadoop.io.compress.DefaultCodec not found.
at
org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:116)
at
org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:156)
at
org.apache.hadoop.mapreduce.lib.input.TextInputFormat.isSplitable(TextInputFormat.java:51)
at
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:254)
at
org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:944)
Aren't these codecs installed by default? If not, how would I enable them?
Thanks
Re: Default Compression
Posted by Mark <st...@gmail.com>.
That did it. Thanks
On 10/31/11 12:52 PM, Joey Echeverria wrote:
> Try getting rid of the extra spaces and new lines.
>
> -Joey
>
> On Mon, Oct 31, 2011 at 1:49 PM, Mark<st...@gmail.com> wrote:
>> I recently added the following to my core-site.xml
>>
>> <property>
>> <name>io.compression.codecs</name>
>> <value>
>> org.apache.hadoop.io.compress.DefaultCodec,
>> org.apache.hadoop.io.compress.GzipCodec,
>> org.apache.hadoop.io.compress.BZip2Codec
>> </value>
>> </property>
>>
>> However when I try and test a simple MR job I am seeing the following errors
>> in my log.
>>
>> java.lang.IllegalArgumentException: Compression codec
>> org.apache.hadoop.io.compress.DefaultCodec not found.
>> at
>> org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:116)
>> at
>> org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:156)
>> at
>> org.apache.hadoop.mapreduce.lib.input.TextInputFormat.isSplitable(TextInputFormat.java:51)
>> at
>> org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:254)
>> at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:944)
>>
>> Aren't these codecs installed by default? If not, how would I enable them?
>>
>> Thanks
>>
>>
>
>
Re: Default Compression
Posted by Joey Echeverria <jo...@cloudera.com>.
Try getting rid of the extra spaces and new lines.
-Joey
On Mon, Oct 31, 2011 at 1:49 PM, Mark <st...@gmail.com> wrote:
> I recently added the following to my core-site.xml
>
> <property>
> <name>io.compression.codecs</name>
> <value>
> org.apache.hadoop.io.compress.DefaultCodec,
> org.apache.hadoop.io.compress.GzipCodec,
> org.apache.hadoop.io.compress.BZip2Codec
> </value>
> </property>
>
> However when I try and test a simple MR job I am seeing the following errors
> in my log.
>
> java.lang.IllegalArgumentException: Compression codec
> org.apache.hadoop.io.compress.DefaultCodec not found.
> at
> org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:116)
> at
> org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:156)
> at
> org.apache.hadoop.mapreduce.lib.input.TextInputFormat.isSplitable(TextInputFormat.java:51)
> at
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:254)
> at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:944)
>
> Aren't these codecs installed by default? If not, how would I enable them?
>
> Thanks
>
>
--
Joseph Echeverria
Cloudera, Inc.
443.305.9434