[jira] [Created] (HADOOP-15776) The key length used in "KeyGenerator.init()" should be configurable

["zhoushulin (JIRA)" <ji...@apache.org>]

zhoushulin created HADOOP-15776:
-----------------------------------

             Summary: The key length used in "KeyGenerator.init()" should be configurable
                 Key: HADOOP-15776
                 URL: https://issues.apache.org/jira/browse/HADOOP-15776
             Project: Hadoop Common
          Issue Type: Bug
          Components: conf
            Reporter: zhoushulin


In mapreduce, the key length used in KeyGenerator.init() is configured with configuration option "mapreduce.job.encrypted-intermediate-data-key-size-bits" as follows:

 
{code:java|title=/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java}
protected void initJobCredentialsAndUGI(Configuration conf) {
  ...
  int keyLen = conf.getInt(
                MRJobConfig.MR_ENCRYPTED_INTERMEDIATE_DATA_KEY_SIZE_BITS,
                MRJobConfig
                        .DEFAULT_MR_ENCRYPTED_INTERMEDIATE_DATA_KEY_SIZE_BITS);
  KeyGenerator keyGen =
                KeyGenerator.getInstance(INTERMEDIATE_DATA_ENCRYPTION_ALGO);
  keyGen.init(keyLen);
  encryptedSpillKey = keyGen.generateKey().getEncoded();
  ...
}
{code}

The same usage is also in mapred as follows:

{code:java|title=/org/apache/hadoop/mapred/LocalJobRunner.java}
public Job(JobID jobid, String jobSubmitDir) throws IOException {  ...
  int keyLen = conf.getInt(
                MRJobConfig.MR_ENCRYPTED_INTERMEDIATE_DATA_KEY_SIZE_BITS,
                MRJobConfig
                        .DEFAULT_MR_ENCRYPTED_INTERMEDIATE_DATA_KEY_SIZE_BITS);
  KeyGenerator keyGen =
                KeyGenerator.getInstance(INTERMEDIATE_DATA_ENCRYPTION_ALGO);
  keyGen.init(keyLen);
  ...
}
{code}


Also, in hadoop-common, there is a configration option "hadoop.security.key.default.bitlength", it is used in KeyProvider.java to initiate KeyGenerator as follows:

{code:java|title=/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java}
public static final String HADOOP_SECURITY_KEY_DEFAULT_BITLENGTH_KEY =
      "hadoop.security.key.default.bitlength";
  /** Defalt value for HADOOP_SECURITY_KEY_DEFAULT_BITLENGTH_KEY. */
  public static final int HADOOP_SECURITY_KEY_DEFAULT_BITLENGTH_DEFAULT = 128;
{code}

{code:java|title=/org/apache/hadoop/crypto/key/KeyProvider.java}
public Options(Configuration conf) {
  cipher = conf.get(DEFAULT_CIPHER_NAME, DEFAULT_CIPHER);
  bitLength = conf.getInt(DEFAULT_BITLENGTH_NAME, DEFAULT_BITLENGTH);
}

public KeyVersion createKey(String name, Options options)
      throws NoSuchAlgorithmException, IOException {
    byte[] material = generateKey(options.getBitLength(), options.getCipher());
    return createKey(name, material, options);
  }

protected byte[] generateKey(int size, String algorithm)
      throws NoSuchAlgorithmException {
    algorithm = getAlgorithm(algorithm);
    KeyGenerator keyGenerator = KeyGenerator.getInstance(algorithm);
    keyGenerator.init(size);
    byte[] key = keyGenerator.generateKey().getEncoded();
    return key;
  }
  ...
}
{code}


However, in other two usage of "KeyGenerator.init()" in mapreduce and hadoop-common, the key length is hard-coded as 64. Also, in the evolving history, this value is changed from "20" to "64". So, in the perspective of flexibility and security, these two hard coded value in "KeyGenerator.init()" should be configurable.

{code:java|title=/org/apache/hadoop/mapreduce/JobSubmitter.java}
class JobSubmitter {
  ...
  private static final int SHUFFLE_KEY_LENGTH = 64;
  ...
  JobStatus submitJobInternal(Job job, Cluster cluster) 
    throws ClassNotFoundException, InterruptedException, IOException {
    ...
    keyGen = KeyGenerator.getInstance(SHUFFLE_KEYGEN_ALGORITHM);
    keyGen.init(SHUFFLE_KEY_LENGTH);
    ...
  }
  ...
}
{code}

{code:java|title=/org/apache/hadoop/security/token/SecretManager.java}
public abstract class SecretManager<T extends TokenIdentifier> {
  ...
  private static final int KEY_LENGTH = 64;
  ...
  private final KeyGenerator keyGen;
  {
    try {
      keyGen = KeyGenerator.getInstance(DEFAULT_HMAC_ALGORITHM);
      keyGen.init(KEY_LENGTH);
    } catch (NoSuchAlgorithmException nsa) {
      throw new IllegalArgumentException("Can't find " + DEFAULT_HMAC_ALGORITHM +
      " algorithm.");
    }
  }
  ...
}
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org