You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Thomas Thevis <th...@vionto.com> on 2009/08/12 11:55:41 UTC
Scheduling dependent jobs with hadoop-0.20.0
Hi all,
currently, we try to upgrade from hadoop-0.19 to hadoop-0.20. Therefore,
we try to get rid of the deprecated JobConf class and switch to
org.apache.hadoop.mapreduce.Job where possible.
In several projects we have chained jobs which depend on each other. For
these, we used the Job and JobControl classes of the
org.apache.hadoop.mapred.jobcontrol package which rely on the deprecated
JobConf class.
How does chaining of dependent jobs work with hadoop-0.20+? Do we still
have to use the org.apache.hadoop.mapred.jobcontrol package or is there
a better way using only the org.apache.hadoop.mapreduce.Job class?
Is there a plan to move org.apache.hadoop.mapred.jobcontrol to a package
org.apache.hadoop.mapreduce.jobcontrol in the future?
Regards,
Thomas
--
Thomas Thevis
Software Developer
------------------------------------------------------------
vionto GmbH
Karl-Marx-Allee 90a, D-10243 Berlin
fon +49 30 40 20 3 29 - 28
fax +49 30 40 20 3 29 - 29
web http://www.vionto.com
------------------------------------------------------------
Geschäftsführer: Ralf von Grafenstein, Dr. Martin C. Hirsch
Sitz der Gesellschaft: Berlin
Amtsgericht Berlin Charlottenburg, HRB 108054B
------------------------------------------------------------
Re: Scheduling dependent jobs with hadoop-0.20.0
Posted by Tom White <to...@cloudera.com>.
Hi Thomas,
The job control feature has been ported to the new MapReduce API in
https://issues.apache.org/jira/browse/MAPREDUCE-368. This will be in
0.21 I think.
Cheers,
Tom
On Wed, Aug 12, 2009 at 10:55 AM, Thomas Thevis<th...@vionto.com> wrote:
> Hi all,
>
> currently, we try to upgrade from hadoop-0.19 to hadoop-0.20. Therefore, we
> try to get rid of the deprecated JobConf class and switch to
> org.apache.hadoop.mapreduce.Job where possible.
> In several projects we have chained jobs which depend on each other. For
> these, we used the Job and JobControl classes of the
> org.apache.hadoop.mapred.jobcontrol package which rely on the deprecated
> JobConf class.
> How does chaining of dependent jobs work with hadoop-0.20+? Do we still have
> to use the org.apache.hadoop.mapred.jobcontrol package or is there a better
> way using only the org.apache.hadoop.mapreduce.Job class?
> Is there a plan to move org.apache.hadoop.mapred.jobcontrol to a package
> org.apache.hadoop.mapreduce.jobcontrol in the future?
>
> Regards,
> Thomas
>
> --
> Thomas Thevis
> Software Developer
> ------------------------------------------------------------
> vionto GmbH
> Karl-Marx-Allee 90a, D-10243 Berlin
>
> fon +49 30 40 20 3 29 - 28
> fax +49 30 40 20 3 29 - 29
> web http://www.vionto.com
> ------------------------------------------------------------
> Geschäftsführer: Ralf von Grafenstein, Dr. Martin C. Hirsch
> Sitz der Gesellschaft: Berlin
> Amtsgericht Berlin Charlottenburg, HRB 108054B
> ------------------------------------------------------------
>