You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@crunch.apache.org by "Igor Bernstein (JIRA)" <ji...@apache.org> on 2016/04/10 21:41:25 UTC

[jira] [Created] (CRUNCH-600) Credentials are lost in local runner mode

Igor Bernstein created CRUNCH-600:
-------------------------------------

             Summary: Credentials are lost in local runner mode
                 Key: CRUNCH-600
                 URL: https://issues.apache.org/jira/browse/CRUNCH-600
             Project: Crunch
          Issue Type: Bug
            Reporter: Igor Bernstein
            Priority: Minor


The credentials attached to a job are lost when CrunchOutputs builds a new Job in getJob.
For example, the following scenario fails:

{code:title=MyTarget.java}
class MyTarget implements MapReduceTarget {
  void configureForMapReduce(Job job, PType<?> ptype, Path outputPath, String name) {
    job.getCredentials().addSecretKey("myPassword", "secret".getBytes());
    CrunchOutputs.addNamedOutput(job, name, FormatBundle.forOutput(MyOutputFormat.class), Object.class);
  }
  //..
}
{code}

{code:title=MyOutputFormat.java}
class MyOutputFormat extends OutputFormat<Object,Object> {
  public RecordWriter<Object, Object> getRecordWriter(TaskAttemptContext context) throws IOException, InterruptedException {
    Assert.equals("secret".getBytes(), context.getCredentials().getSecretKey("myPassword"));
  }
}
{code}

This happens because CrunchOutputs.getJob copies the baseConf using the base Configuration class and the base Configuration class doesn't have a notion of Credentials, so the credentials get dropped.

This is issue only manifests itself when using the local runner. In an actual hadoop cluster the credentials will be inherited from UserGroupInformation.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)