JobDebugger bug / wrong Hadoop version

[Philip Tromans <ph...@gmail.com>]

Hi all,

I'm running into a problem - I'm using Hive trunk (pretty recent, but
I see the bug's in trunk at time of writing as well), with Hadoop
0.20.205.0. I have a job which fails (for a reason which is entirely
my own fault), and when it does fail Hive dies with the following
exception:

Ended Job = job_201202291327_1399 with errors
Error during job, obtaining debugging information...
Examining task ID: task_201202291327_1399_m_000003 (and more) from job
job_201202291327_1399
Examining task ID: task_201202291327_1399_r_000002 (and more) from job
job_201202291327_1399
Exception in thread "Thread-342" java.lang.RuntimeException: Error
while reading from task log url
        at org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:130)
        at org.apache.hadoop.hive.ql.exec.JobDebugger.showJobFailDebugInfo(JobDebugger.java:211)
        at org.apache.hadoop.hive.ql.exec.JobDebugger.run(JobDebugger.java:81)
        at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: Server returned HTTP response code:
400 for URL: http://...:50060/tasklog?taskid=attempt_201202291327_1399_r_000002_2&start=-8193
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1436)
        at java.net.URL.openStream(URL.java:1010)
        at org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:120)
        ... 3 more

When I point a web browser at the given URL, I get the following message:

HTTP ERROR 400

Problem accessing /tasklog. Reason:

    Argument attemptid is required

If I change taskid to attemptid, everything works perfectly. The code
which generates this URL appears to be in
org.apache.hadoop.hive.ql.exec.JobDebugger.java. I presume that this
code is correct for a given version of Hadoop. Which version is
currently in use in the Jenkins/Hudson build environment? I'd be happy
to change it and submit a patch to JIRA, but I guess that'd probably
break the other version of Hadoop, so perhaps some more profound
versioning type thing might be needed.

This is the issue that HIVE-1579 is referring to.

Cheers,

Phil.