killJob method in JobClient

[Robert Goodman <bs...@gmail.com>]

I have some code where I create my own Hadoop job and the use the JobClient
to submit the Hadoop job. I noticed that the JobClient class has a
killJob() method. I was planning to play around and try to kill a running
Hadoop job. Does anybody know status the killJob method? I'm using Hadoop
0.17.2.1 My concern is that it might work in some phases (map, reduce, etc)
of a Hadoop job, but not in other phases which is hard to find with simple
testing. There may be hidden gotchas that are hard to detect. This concern
is really driven by the fact that the jobtracker.jsp in Hadoop doesn't have
an option to kill a job. This would be a useful option.

   Thanks
    Bob

Re: killJob method in JobClient

[Steve Loughran <st...@apache.org>]

Robert Goodman wrote:
> I have some code where I create my own Hadoop job and the use the JobClient
> to submit the Hadoop job. I noticed that the JobClient class has a
> killJob() method. I was planning to play around and try to kill a running
> Hadoop job. Does anybody know status the killJob method? I'm using Hadoop
> 0.17.2.1 My concern is that it might work in some phases (map, reduce, etc)
> of a Hadoop job, but not in other phases which is hard to find with simple
> testing. There may be hidden gotchas that are hard to detect. This concern
> is really driven by the fact that the jobtracker.jsp in Hadoop doesn't have
> an option to kill a job. This would be a useful option.
> 

If you get your API operations added as more functional tests to the 
hadoop codebase, then you can be reasonably confident that nobody is 
going to break your code. This is the best way of ensuring that your 
system remains supported -make it part of the test suite

-steve

Re: killJob method in JobClient

[Jason Venner <ja...@attributor.com>]

The jobtracker does have the option to kill a job if you enable it in 
the hadoop-site.xml file:

<property>
  <name>webinterface.private.actions</name>
  <value>true</value>
  <description> If set to true, the web interfaces of JT and NN may contain
                actions, such as kill job, delete file, etc., that should
                not be exposed to public. Enable this option if the 
interfaces
                are only reachable by those who have the right 
authorization.
  </description>
</property>

Also the bin/hadoop job -kill and bin/hadoop job -list commands may be 
of interest to you.


Robert Goodman wrote:
> I have some code where I create my own Hadoop job and the use the JobClient
> to submit the Hadoop job. I noticed that the JobClient class has a
> killJob() method. I was planning to play around and try to kill a running
> Hadoop job. Does anybody know status the killJob method? I'm using Hadoop
> 0.17.2.1 My concern is that it might work in some phases (map, reduce, etc)
> of a Hadoop job, but not in other phases which is hard to find with simple
> testing. There may be hidden gotchas that are hard to detect. This concern
> is really driven by the fact that the jobtracker.jsp in Hadoop doesn't have
> an option to kill a job. This would be a useful option.
>
>    Thanks
>     Bob
>
>   
-- 
Jason Venner
Attributor - Program the Web <http://www.attributor.com/>