[jira] Commented: (HADOOP-1851) Map output compression coded cannot be set independently of job output compression codec

["Riccardo (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-1851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12525535 ] 

Riccardo commented on HADOOP-1851:
----------------------------------

The methods that affect this behavior are in line 341-371 of JobConf in Hadoop 0.14.1:

/**
   * Set the given class as the  compression codec for the map outputs.
   * @param codecClass the CompressionCodec class that will compress the 
   *                   map outputs
   */
  public void setMapOutputCompressorClass(Class<? extends CompressionCodec> codecClass) {
    setCompressMapOutput(true);
    setClass("mapred.output.compression.codec", codecClass, 
             CompressionCodec.class);
  }
  
  /**
   * Get the codec for compressing the map outputs
   * @param defaultValue the value to return if it is not set
   * @return the CompressionCodec class that should be used to compress the 
   *   map outputs
   * @throws IllegalArgumentException if the class was specified, but not found
   */
  public Class<? extends CompressionCodec> getMapOutputCompressorClass(Class<? extends CompressionCodec> defaultValue) {
    String name = get("mapred.output.compression.codec");
    if (name == null) {
      return defaultValue;
    } else {
      try {
        return getClassByName(name).asSubclass(CompressionCodec.class);
      } catch (ClassNotFoundException e) {
        throw new IllegalArgumentException("Compression codec " + name + 
                                           " was not found.", e);
      }
    }
  }

This could be easily fixed by using a different property, for example, "map.output.compression.codec".

> Map output compression coded cannot be set independently of job output compression codec
> ----------------------------------------------------------------------------------------
>
>                 Key: HADOOP-1851
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1851
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.14.1
>            Reporter: Riccardo
>
> The property "mapred.output.compression.codec" is used when setting and getting the map output compression codec in JobConf, thus making it impossible to use a different codec for map outputs and overall job outputs.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.