You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by Jie Li <ji...@cs.duke.edu> on 2012/08/16 02:47:45 UTC

Hadoop profiling will fail when Kerberos is enabled

Hi all,

With Kerbero enabled, the job client is not able to download profiles from
the tasktracker. The following command will fail:

hadoop jar hadoop-examples-1.0.2.jar wordcount -Dmapred.task.profile=true
input output

The error message:

12/08/16 00:15:37 INFO mapred.JobClient: Running job: job_201208152133_0007
12/08/16 00:15:38 INFO mapred.JobClient:  map 0% reduce 0%
java.io.IOException: Server returned HTTP response code: 401 for URL:
http://datanode:50060/tasklog?plaintext=true&attemptid=attempt_201208152133_0007_m_000002_0&filter=profile<http://esv4-hcl266.grid.linkedin.com:50060/tasklog?plaintext=true&attemptid=attempt_201208152133_0007_m_000002_0&filter=profile>
    at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1436)
    at
org.apache.hadoop.mapred.JobClient.downloadProfile(JobClient.java:1237)
    at
org.apache.hadoop.mapred.JobClient.monitorAndPrintJob(JobClient.java:1315)
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:533)
    at org.apache.hadoop.examples.WordCount.main(WordCount.java:67)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
    at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
    at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

Note if we copy the URL link above to the browser, we are able to see the
profile output. Any hint is appreciated!

Thanks,
Jie

Re: Hadoop profiling will fail when Kerberos is enabled

Posted by Hache Bass <tr...@yahoo.com>.

Jie Li, Did you ever figure out how to solve this?

Thanks,
Hache

Re: Hadoop profiling will fail when Kerberos is enabled

Posted by Hache Bass <tr...@yahoo.com>.

Jie Li, Did you ever figure out how to solve this?

Thanks,
Hache

Re: Hadoop profiling will fail when Kerberos is enabled

Posted by Hache Bass <tr...@yahoo.com>.

Jie Li, Did you ever figure out how to solve this?

Thanks,
Hache

Re: Hadoop profiling will fail when Kerberos is enabled

Posted by Hache Bass <tr...@yahoo.com>.

Jie Li, Did you ever figure out how to solve this?

Thanks,
Hache