You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Hansi Klose <ha...@web.de> on 2013/12/03 17:38:47 UTC
mapreduce.jobtracker.expire.trackers.interval no effect
Hi,
we want to set the heartbeat timout for a tasktracker.
If the tasktracker does not send heartbeats for 60 seconds he should
be marked as lost.
I found the parameter mapreduce.jobtracker.expire.trackers.interval
which sounds right to me.
I set
<property>
<name>mapreduce.jobtracker.expire.trackers.interval</name>
<value>60000</value>
</property>
in the mapred-site.xml on all servers and restarted the jobtracker and all tasktrackers.
I started a benchmark "hadoop jar hadoop-examples.jar randomwriter rand" and every tasktracker gets 2 jobs.
It is a small test environment.
On one tasktracker i stopped the network. On the jobtracker i could see the "Seconds since heartbeat"
increasing. But after 60 seconds the tasktracker was still in the overview.
Even in the log of the jobtracker I found nothing.
After over 600 seconds i found the message
org.apache.hadoop.mapred.JobTracker: Lost tracker .....
And the tasktracker wasn't shown any more on the jobtracker.
Isn't this the right setting?
Regards Hansi
Aw: Re: mapreduce.jobtracker.expire.trackers.interval no effect
Posted by Hansi Klose <ha...@web.de>.
Hi adam,
in our enviroment it does not matter what i insert there it always take over 600 seconds.
I tried 30000 and the resulte was the same.
Regards Hansi
Gesendet: Dienstag, 03. Dezember 2013 um 19:23 Uhr
Von: "Adam Kawa" <ka...@gmail.com>
An: user@hadoop.apache.org
Betreff: Re: mapreduce.jobtracker.expire.trackers.interval no effect
I did a small test, and I a setting mapred.tasktracker.expiry.interval=60000 worked for me (TT became considered as lost after around 66 seconds).
Can the formula be: mapred.tasktracker.expiry.interval + 2 * some-heartbeat-interval-that-is-3-sec-by-default?
Otherwise, is the 6 sec some kind of time needed to make a decision to consider TT as lost?
2013/12/3 Hansi Klose <ha...@web.de>I forget to say that we use Cloudera 2.0.0-mr1-cdh4.2.0
> Gesendet: Dienstag, 03. Dezember 2013 um 17:38 Uhr
> Von: "Hansi Klose" <hansi.klose@web.de[hansi.klose@web.de]>
> An: user@hadoop.apache.org[user@hadoop.apache.org]
> Betreff: mapreduce.jobtracker.expire.trackers.interval no effect
>
> Hi,
>
> we want to set the heartbeat timout for a tasktracker.
>
> If the tasktracker does not send heartbeats for 60 seconds he should
> be marked as lost.
>
> I found the parameter mapreduce.jobtracker.expire.trackers.interval
> which sounds right to me.
>
> I set
>
> <property>
> <name>mapreduce.jobtracker.expire.trackers.interval</name>
> <value>60000</value>
> </property>
>
> in the mapred-site.xml on all servers and restarted the jobtracker and all tasktrackers.
>
> I started a benchmark "hadoop jar hadoop-examples.jar randomwriter rand" and every tasktracker gets 2 jobs.
> It is a small test environment.
>
> On one tasktracker i stopped the network. On the jobtracker i could see the "Seconds since heartbeat"
> increasing. But after 60 seconds the tasktracker was still in the overview.
> Even in the log of the jobtracker I found nothing.
>
> After over 600 seconds i found the message
> org.apache.hadoop.mapred.JobTracker: Lost tracker .....
> And the tasktracker wasn't shown any more on the jobtracker.
>
> Isn't this the right setting?
>
> Regards Hansi
>
Aw: Re: mapreduce.jobtracker.expire.trackers.interval no effect
Posted by Hansi Klose <ha...@web.de>.
Hi adam,
in our enviroment it does not matter what i insert there it always take over 600 seconds.
I tried 30000 and the resulte was the same.
Regards Hansi
Gesendet: Dienstag, 03. Dezember 2013 um 19:23 Uhr
Von: "Adam Kawa" <ka...@gmail.com>
An: user@hadoop.apache.org
Betreff: Re: mapreduce.jobtracker.expire.trackers.interval no effect
I did a small test, and I a setting mapred.tasktracker.expiry.interval=60000 worked for me (TT became considered as lost after around 66 seconds).
Can the formula be: mapred.tasktracker.expiry.interval + 2 * some-heartbeat-interval-that-is-3-sec-by-default?
Otherwise, is the 6 sec some kind of time needed to make a decision to consider TT as lost?
2013/12/3 Hansi Klose <ha...@web.de>I forget to say that we use Cloudera 2.0.0-mr1-cdh4.2.0
> Gesendet: Dienstag, 03. Dezember 2013 um 17:38 Uhr
> Von: "Hansi Klose" <hansi.klose@web.de[hansi.klose@web.de]>
> An: user@hadoop.apache.org[user@hadoop.apache.org]
> Betreff: mapreduce.jobtracker.expire.trackers.interval no effect
>
> Hi,
>
> we want to set the heartbeat timout for a tasktracker.
>
> If the tasktracker does not send heartbeats for 60 seconds he should
> be marked as lost.
>
> I found the parameter mapreduce.jobtracker.expire.trackers.interval
> which sounds right to me.
>
> I set
>
> <property>
> <name>mapreduce.jobtracker.expire.trackers.interval</name>
> <value>60000</value>
> </property>
>
> in the mapred-site.xml on all servers and restarted the jobtracker and all tasktrackers.
>
> I started a benchmark "hadoop jar hadoop-examples.jar randomwriter rand" and every tasktracker gets 2 jobs.
> It is a small test environment.
>
> On one tasktracker i stopped the network. On the jobtracker i could see the "Seconds since heartbeat"
> increasing. But after 60 seconds the tasktracker was still in the overview.
> Even in the log of the jobtracker I found nothing.
>
> After over 600 seconds i found the message
> org.apache.hadoop.mapred.JobTracker: Lost tracker .....
> And the tasktracker wasn't shown any more on the jobtracker.
>
> Isn't this the right setting?
>
> Regards Hansi
>
Aw: Re: mapreduce.jobtracker.expire.trackers.interval no effect
Posted by Hansi Klose <ha...@web.de>.
Hi adam,
in our enviroment it does not matter what i insert there it always take over 600 seconds.
I tried 30000 and the resulte was the same.
Regards Hansi
Gesendet: Dienstag, 03. Dezember 2013 um 19:23 Uhr
Von: "Adam Kawa" <ka...@gmail.com>
An: user@hadoop.apache.org
Betreff: Re: mapreduce.jobtracker.expire.trackers.interval no effect
I did a small test, and I a setting mapred.tasktracker.expiry.interval=60000 worked for me (TT became considered as lost after around 66 seconds).
Can the formula be: mapred.tasktracker.expiry.interval + 2 * some-heartbeat-interval-that-is-3-sec-by-default?
Otherwise, is the 6 sec some kind of time needed to make a decision to consider TT as lost?
2013/12/3 Hansi Klose <ha...@web.de>I forget to say that we use Cloudera 2.0.0-mr1-cdh4.2.0
> Gesendet: Dienstag, 03. Dezember 2013 um 17:38 Uhr
> Von: "Hansi Klose" <hansi.klose@web.de[hansi.klose@web.de]>
> An: user@hadoop.apache.org[user@hadoop.apache.org]
> Betreff: mapreduce.jobtracker.expire.trackers.interval no effect
>
> Hi,
>
> we want to set the heartbeat timout for a tasktracker.
>
> If the tasktracker does not send heartbeats for 60 seconds he should
> be marked as lost.
>
> I found the parameter mapreduce.jobtracker.expire.trackers.interval
> which sounds right to me.
>
> I set
>
> <property>
> <name>mapreduce.jobtracker.expire.trackers.interval</name>
> <value>60000</value>
> </property>
>
> in the mapred-site.xml on all servers and restarted the jobtracker and all tasktrackers.
>
> I started a benchmark "hadoop jar hadoop-examples.jar randomwriter rand" and every tasktracker gets 2 jobs.
> It is a small test environment.
>
> On one tasktracker i stopped the network. On the jobtracker i could see the "Seconds since heartbeat"
> increasing. But after 60 seconds the tasktracker was still in the overview.
> Even in the log of the jobtracker I found nothing.
>
> After over 600 seconds i found the message
> org.apache.hadoop.mapred.JobTracker: Lost tracker .....
> And the tasktracker wasn't shown any more on the jobtracker.
>
> Isn't this the right setting?
>
> Regards Hansi
>
Aw: Re: mapreduce.jobtracker.expire.trackers.interval no effect
Posted by Hansi Klose <ha...@web.de>.
Hi adam,
in our enviroment it does not matter what i insert there it always take over 600 seconds.
I tried 30000 and the resulte was the same.
Regards Hansi
Gesendet: Dienstag, 03. Dezember 2013 um 19:23 Uhr
Von: "Adam Kawa" <ka...@gmail.com>
An: user@hadoop.apache.org
Betreff: Re: mapreduce.jobtracker.expire.trackers.interval no effect
I did a small test, and I a setting mapred.tasktracker.expiry.interval=60000 worked for me (TT became considered as lost after around 66 seconds).
Can the formula be: mapred.tasktracker.expiry.interval + 2 * some-heartbeat-interval-that-is-3-sec-by-default?
Otherwise, is the 6 sec some kind of time needed to make a decision to consider TT as lost?
2013/12/3 Hansi Klose <ha...@web.de>I forget to say that we use Cloudera 2.0.0-mr1-cdh4.2.0
> Gesendet: Dienstag, 03. Dezember 2013 um 17:38 Uhr
> Von: "Hansi Klose" <hansi.klose@web.de[hansi.klose@web.de]>
> An: user@hadoop.apache.org[user@hadoop.apache.org]
> Betreff: mapreduce.jobtracker.expire.trackers.interval no effect
>
> Hi,
>
> we want to set the heartbeat timout for a tasktracker.
>
> If the tasktracker does not send heartbeats for 60 seconds he should
> be marked as lost.
>
> I found the parameter mapreduce.jobtracker.expire.trackers.interval
> which sounds right to me.
>
> I set
>
> <property>
> <name>mapreduce.jobtracker.expire.trackers.interval</name>
> <value>60000</value>
> </property>
>
> in the mapred-site.xml on all servers and restarted the jobtracker and all tasktrackers.
>
> I started a benchmark "hadoop jar hadoop-examples.jar randomwriter rand" and every tasktracker gets 2 jobs.
> It is a small test environment.
>
> On one tasktracker i stopped the network. On the jobtracker i could see the "Seconds since heartbeat"
> increasing. But after 60 seconds the tasktracker was still in the overview.
> Even in the log of the jobtracker I found nothing.
>
> After over 600 seconds i found the message
> org.apache.hadoop.mapred.JobTracker: Lost tracker .....
> And the tasktracker wasn't shown any more on the jobtracker.
>
> Isn't this the right setting?
>
> Regards Hansi
>
Re: mapreduce.jobtracker.expire.trackers.interval no effect
Posted by Adam Kawa <ka...@gmail.com>.
I did a small test, and I a setting
mapred.tasktracker.expiry.interval=60000 worked
for me (TT became considered as lost after around 66 seconds).
Can the formula be: mapred.tasktracker.expiry.interval + 2 *
some-heartbeat-interval-that-is-3-sec-by-default?
Otherwise, is the 6 sec some kind of time needed to make a decision to
consider TT as lost?
2013/12/3 Hansi Klose <ha...@web.de>
> I forget to say that we use Cloudera 2.0.0-mr1-cdh4.2.0
>
> > Gesendet: Dienstag, 03. Dezember 2013 um 17:38 Uhr
> > Von: "Hansi Klose" <ha...@web.de>
> > An: user@hadoop.apache.org
> > Betreff: mapreduce.jobtracker.expire.trackers.interval no effect
> >
> > Hi,
> >
> > we want to set the heartbeat timout for a tasktracker.
> >
> > If the tasktracker does not send heartbeats for 60 seconds he should
> > be marked as lost.
> >
> > I found the parameter mapreduce.jobtracker.expire.trackers.interval
> > which sounds right to me.
> >
> > I set
> >
> > <property>
> > <name>mapreduce.jobtracker.expire.trackers.interval</name>
> > <value>60000</value>
> > </property>
> >
> > in the mapred-site.xml on all servers and restarted the jobtracker and
> all tasktrackers.
> >
> > I started a benchmark "hadoop jar hadoop-examples.jar randomwriter rand"
> and every tasktracker gets 2 jobs.
> > It is a small test environment.
> >
> > On one tasktracker i stopped the network. On the jobtracker i could see
> the "Seconds since heartbeat"
> > increasing. But after 60 seconds the tasktracker was still in the
> overview.
> > Even in the log of the jobtracker I found nothing.
> >
> > After over 600 seconds i found the message
> > org.apache.hadoop.mapred.JobTracker: Lost tracker .....
> > And the tasktracker wasn't shown any more on the jobtracker.
> >
> > Isn't this the right setting?
> >
> > Regards Hansi
> >
>
Re: mapreduce.jobtracker.expire.trackers.interval no effect
Posted by Adam Kawa <ka...@gmail.com>.
I did a small test, and I a setting
mapred.tasktracker.expiry.interval=60000 worked
for me (TT became considered as lost after around 66 seconds).
Can the formula be: mapred.tasktracker.expiry.interval + 2 *
some-heartbeat-interval-that-is-3-sec-by-default?
Otherwise, is the 6 sec some kind of time needed to make a decision to
consider TT as lost?
2013/12/3 Hansi Klose <ha...@web.de>
> I forget to say that we use Cloudera 2.0.0-mr1-cdh4.2.0
>
> > Gesendet: Dienstag, 03. Dezember 2013 um 17:38 Uhr
> > Von: "Hansi Klose" <ha...@web.de>
> > An: user@hadoop.apache.org
> > Betreff: mapreduce.jobtracker.expire.trackers.interval no effect
> >
> > Hi,
> >
> > we want to set the heartbeat timout for a tasktracker.
> >
> > If the tasktracker does not send heartbeats for 60 seconds he should
> > be marked as lost.
> >
> > I found the parameter mapreduce.jobtracker.expire.trackers.interval
> > which sounds right to me.
> >
> > I set
> >
> > <property>
> > <name>mapreduce.jobtracker.expire.trackers.interval</name>
> > <value>60000</value>
> > </property>
> >
> > in the mapred-site.xml on all servers and restarted the jobtracker and
> all tasktrackers.
> >
> > I started a benchmark "hadoop jar hadoop-examples.jar randomwriter rand"
> and every tasktracker gets 2 jobs.
> > It is a small test environment.
> >
> > On one tasktracker i stopped the network. On the jobtracker i could see
> the "Seconds since heartbeat"
> > increasing. But after 60 seconds the tasktracker was still in the
> overview.
> > Even in the log of the jobtracker I found nothing.
> >
> > After over 600 seconds i found the message
> > org.apache.hadoop.mapred.JobTracker: Lost tracker .....
> > And the tasktracker wasn't shown any more on the jobtracker.
> >
> > Isn't this the right setting?
> >
> > Regards Hansi
> >
>
Re: mapreduce.jobtracker.expire.trackers.interval no effect
Posted by Adam Kawa <ka...@gmail.com>.
I did a small test, and I a setting
mapred.tasktracker.expiry.interval=60000 worked
for me (TT became considered as lost after around 66 seconds).
Can the formula be: mapred.tasktracker.expiry.interval + 2 *
some-heartbeat-interval-that-is-3-sec-by-default?
Otherwise, is the 6 sec some kind of time needed to make a decision to
consider TT as lost?
2013/12/3 Hansi Klose <ha...@web.de>
> I forget to say that we use Cloudera 2.0.0-mr1-cdh4.2.0
>
> > Gesendet: Dienstag, 03. Dezember 2013 um 17:38 Uhr
> > Von: "Hansi Klose" <ha...@web.de>
> > An: user@hadoop.apache.org
> > Betreff: mapreduce.jobtracker.expire.trackers.interval no effect
> >
> > Hi,
> >
> > we want to set the heartbeat timout for a tasktracker.
> >
> > If the tasktracker does not send heartbeats for 60 seconds he should
> > be marked as lost.
> >
> > I found the parameter mapreduce.jobtracker.expire.trackers.interval
> > which sounds right to me.
> >
> > I set
> >
> > <property>
> > <name>mapreduce.jobtracker.expire.trackers.interval</name>
> > <value>60000</value>
> > </property>
> >
> > in the mapred-site.xml on all servers and restarted the jobtracker and
> all tasktrackers.
> >
> > I started a benchmark "hadoop jar hadoop-examples.jar randomwriter rand"
> and every tasktracker gets 2 jobs.
> > It is a small test environment.
> >
> > On one tasktracker i stopped the network. On the jobtracker i could see
> the "Seconds since heartbeat"
> > increasing. But after 60 seconds the tasktracker was still in the
> overview.
> > Even in the log of the jobtracker I found nothing.
> >
> > After over 600 seconds i found the message
> > org.apache.hadoop.mapred.JobTracker: Lost tracker .....
> > And the tasktracker wasn't shown any more on the jobtracker.
> >
> > Isn't this the right setting?
> >
> > Regards Hansi
> >
>
Re: mapreduce.jobtracker.expire.trackers.interval no effect
Posted by Adam Kawa <ka...@gmail.com>.
I did a small test, and I a setting
mapred.tasktracker.expiry.interval=60000 worked
for me (TT became considered as lost after around 66 seconds).
Can the formula be: mapred.tasktracker.expiry.interval + 2 *
some-heartbeat-interval-that-is-3-sec-by-default?
Otherwise, is the 6 sec some kind of time needed to make a decision to
consider TT as lost?
2013/12/3 Hansi Klose <ha...@web.de>
> I forget to say that we use Cloudera 2.0.0-mr1-cdh4.2.0
>
> > Gesendet: Dienstag, 03. Dezember 2013 um 17:38 Uhr
> > Von: "Hansi Klose" <ha...@web.de>
> > An: user@hadoop.apache.org
> > Betreff: mapreduce.jobtracker.expire.trackers.interval no effect
> >
> > Hi,
> >
> > we want to set the heartbeat timout for a tasktracker.
> >
> > If the tasktracker does not send heartbeats for 60 seconds he should
> > be marked as lost.
> >
> > I found the parameter mapreduce.jobtracker.expire.trackers.interval
> > which sounds right to me.
> >
> > I set
> >
> > <property>
> > <name>mapreduce.jobtracker.expire.trackers.interval</name>
> > <value>60000</value>
> > </property>
> >
> > in the mapred-site.xml on all servers and restarted the jobtracker and
> all tasktrackers.
> >
> > I started a benchmark "hadoop jar hadoop-examples.jar randomwriter rand"
> and every tasktracker gets 2 jobs.
> > It is a small test environment.
> >
> > On one tasktracker i stopped the network. On the jobtracker i could see
> the "Seconds since heartbeat"
> > increasing. But after 60 seconds the tasktracker was still in the
> overview.
> > Even in the log of the jobtracker I found nothing.
> >
> > After over 600 seconds i found the message
> > org.apache.hadoop.mapred.JobTracker: Lost tracker .....
> > And the tasktracker wasn't shown any more on the jobtracker.
> >
> > Isn't this the right setting?
> >
> > Regards Hansi
> >
>
Aw: mapreduce.jobtracker.expire.trackers.interval no effect
Posted by Hansi Klose <ha...@web.de>.
I forget to say that we use Cloudera 2.0.0-mr1-cdh4.2.0
> Gesendet: Dienstag, 03. Dezember 2013 um 17:38 Uhr
> Von: "Hansi Klose" <ha...@web.de>
> An: user@hadoop.apache.org
> Betreff: mapreduce.jobtracker.expire.trackers.interval no effect
>
> Hi,
>
> we want to set the heartbeat timout for a tasktracker.
>
> If the tasktracker does not send heartbeats for 60 seconds he should
> be marked as lost.
>
> I found the parameter mapreduce.jobtracker.expire.trackers.interval
> which sounds right to me.
>
> I set
>
> <property>
> <name>mapreduce.jobtracker.expire.trackers.interval</name>
> <value>60000</value>
> </property>
>
> in the mapred-site.xml on all servers and restarted the jobtracker and all tasktrackers.
>
> I started a benchmark "hadoop jar hadoop-examples.jar randomwriter rand" and every tasktracker gets 2 jobs.
> It is a small test environment.
>
> On one tasktracker i stopped the network. On the jobtracker i could see the "Seconds since heartbeat"
> increasing. But after 60 seconds the tasktracker was still in the overview.
> Even in the log of the jobtracker I found nothing.
>
> After over 600 seconds i found the message
> org.apache.hadoop.mapred.JobTracker: Lost tracker .....
> And the tasktracker wasn't shown any more on the jobtracker.
>
> Isn't this the right setting?
>
> Regards Hansi
>
Aw: mapreduce.jobtracker.expire.trackers.interval no effect
Posted by Hansi Klose <ha...@web.de>.
I forget to say that we use Cloudera 2.0.0-mr1-cdh4.2.0
> Gesendet: Dienstag, 03. Dezember 2013 um 17:38 Uhr
> Von: "Hansi Klose" <ha...@web.de>
> An: user@hadoop.apache.org
> Betreff: mapreduce.jobtracker.expire.trackers.interval no effect
>
> Hi,
>
> we want to set the heartbeat timout for a tasktracker.
>
> If the tasktracker does not send heartbeats for 60 seconds he should
> be marked as lost.
>
> I found the parameter mapreduce.jobtracker.expire.trackers.interval
> which sounds right to me.
>
> I set
>
> <property>
> <name>mapreduce.jobtracker.expire.trackers.interval</name>
> <value>60000</value>
> </property>
>
> in the mapred-site.xml on all servers and restarted the jobtracker and all tasktrackers.
>
> I started a benchmark "hadoop jar hadoop-examples.jar randomwriter rand" and every tasktracker gets 2 jobs.
> It is a small test environment.
>
> On one tasktracker i stopped the network. On the jobtracker i could see the "Seconds since heartbeat"
> increasing. But after 60 seconds the tasktracker was still in the overview.
> Even in the log of the jobtracker I found nothing.
>
> After over 600 seconds i found the message
> org.apache.hadoop.mapred.JobTracker: Lost tracker .....
> And the tasktracker wasn't shown any more on the jobtracker.
>
> Isn't this the right setting?
>
> Regards Hansi
>
Re: mapreduce.jobtracker.expire.trackers.interval no effect
Posted by Adam Kawa <ka...@gmail.com>.
> So i tried the deprecated parameter mapred.tasktracker.expiry.interval in
> my configuration and voila it works!
>
Hansi, this is exactly the one parameter that I told you about in a
previous post ;)
Re: mapreduce.jobtracker.expire.trackers.interval no effect
Posted by Adam Kawa <ka...@gmail.com>.
> So i tried the deprecated parameter mapred.tasktracker.expiry.interval in
> my configuration and voila it works!
>
Hansi, this is exactly the one parameter that I told you about in a
previous post ;)
Re: mapreduce.jobtracker.expire.trackers.interval no effect
Posted by Adam Kawa <ka...@gmail.com>.
> So i tried the deprecated parameter mapred.tasktracker.expiry.interval in
> my configuration and voila it works!
>
Hansi, this is exactly the one parameter that I told you about in a
previous post ;)
Re: mapreduce.jobtracker.expire.trackers.interval no effect
Posted by Adam Kawa <ka...@gmail.com>.
> So i tried the deprecated parameter mapred.tasktracker.expiry.interval in
> my configuration and voila it works!
>
Hansi, this is exactly the one parameter that I told you about in a
previous post ;)
Aw: mapreduce.jobtracker.expire.trackers.interval no effect
Posted by Hansi Klose <ha...@web.de>.
Hi.
i think i found the reason. I looked at the job.xml and found the parameter
mapred.tasktracker.expiry.interval 6000000
and
mapreduce.jobtracker.expire.trackers.interval 30000
So i tried the deprecated parameter mapred.tasktracker.expiry.interval in my configuration and voila it works!
Why they write that the parameter is deprecated when the new one is not working and will be overwritten by the old one
with the default value?
http://archive.cloudera.com/cdh4/cdh/4/hadoop-2.0.0-cdh4.2.0/hadoop-project-dist/hadoop-common/DeprecatedProperties.html
Regards Hansi
> Gesendet: Dienstag, 03. Dezember 2013 um 17:38 Uhr
> Von: "Hansi Klose" <ha...@web.de>
> An: user@hadoop.apache.org
> Betreff: mapreduce.jobtracker.expire.trackers.interval no effect
>
> Hi,
>
> we want to set the heartbeat timout for a tasktracker.
>
> If the tasktracker does not send heartbeats for 60 seconds he should
> be marked as lost.
>
> I found the parameter mapreduce.jobtracker.expire.trackers.interval
> which sounds right to me.
>
> I set
>
> <property>
> <name>mapreduce.jobtracker.expire.trackers.interval</name>
> <value>60000</value>
> </property>
>
> in the mapred-site.xml on all servers and restarted the jobtracker and all tasktrackers.
>
> I started a benchmark "hadoop jar hadoop-examples.jar randomwriter rand" and every tasktracker gets 2 jobs.
> It is a small test environment.
>
> On one tasktracker i stopped the network. On the jobtracker i could see the "Seconds since heartbeat"
> increasing. But after 60 seconds the tasktracker was still in the overview.
> Even in the log of the jobtracker I found nothing.
>
> After over 600 seconds i found the message
> org.apache.hadoop.mapred.JobTracker: Lost tracker .....
> And the tasktracker wasn't shown any more on the jobtracker.
>
> Isn't this the right setting?
>
> Regards Hansi
>
Aw: mapreduce.jobtracker.expire.trackers.interval no effect
Posted by Hansi Klose <ha...@web.de>.
I forget to say that we use Cloudera 2.0.0-mr1-cdh4.2.0
> Gesendet: Dienstag, 03. Dezember 2013 um 17:38 Uhr
> Von: "Hansi Klose" <ha...@web.de>
> An: user@hadoop.apache.org
> Betreff: mapreduce.jobtracker.expire.trackers.interval no effect
>
> Hi,
>
> we want to set the heartbeat timout for a tasktracker.
>
> If the tasktracker does not send heartbeats for 60 seconds he should
> be marked as lost.
>
> I found the parameter mapreduce.jobtracker.expire.trackers.interval
> which sounds right to me.
>
> I set
>
> <property>
> <name>mapreduce.jobtracker.expire.trackers.interval</name>
> <value>60000</value>
> </property>
>
> in the mapred-site.xml on all servers and restarted the jobtracker and all tasktrackers.
>
> I started a benchmark "hadoop jar hadoop-examples.jar randomwriter rand" and every tasktracker gets 2 jobs.
> It is a small test environment.
>
> On one tasktracker i stopped the network. On the jobtracker i could see the "Seconds since heartbeat"
> increasing. But after 60 seconds the tasktracker was still in the overview.
> Even in the log of the jobtracker I found nothing.
>
> After over 600 seconds i found the message
> org.apache.hadoop.mapred.JobTracker: Lost tracker .....
> And the tasktracker wasn't shown any more on the jobtracker.
>
> Isn't this the right setting?
>
> Regards Hansi
>
Aw: mapreduce.jobtracker.expire.trackers.interval no effect
Posted by Hansi Klose <ha...@web.de>.
Hi.
i think i found the reason. I looked at the job.xml and found the parameter
mapred.tasktracker.expiry.interval 6000000
and
mapreduce.jobtracker.expire.trackers.interval 30000
So i tried the deprecated parameter mapred.tasktracker.expiry.interval in my configuration and voila it works!
Why they write that the parameter is deprecated when the new one is not working and will be overwritten by the old one
with the default value?
http://archive.cloudera.com/cdh4/cdh/4/hadoop-2.0.0-cdh4.2.0/hadoop-project-dist/hadoop-common/DeprecatedProperties.html
Regards Hansi
> Gesendet: Dienstag, 03. Dezember 2013 um 17:38 Uhr
> Von: "Hansi Klose" <ha...@web.de>
> An: user@hadoop.apache.org
> Betreff: mapreduce.jobtracker.expire.trackers.interval no effect
>
> Hi,
>
> we want to set the heartbeat timout for a tasktracker.
>
> If the tasktracker does not send heartbeats for 60 seconds he should
> be marked as lost.
>
> I found the parameter mapreduce.jobtracker.expire.trackers.interval
> which sounds right to me.
>
> I set
>
> <property>
> <name>mapreduce.jobtracker.expire.trackers.interval</name>
> <value>60000</value>
> </property>
>
> in the mapred-site.xml on all servers and restarted the jobtracker and all tasktrackers.
>
> I started a benchmark "hadoop jar hadoop-examples.jar randomwriter rand" and every tasktracker gets 2 jobs.
> It is a small test environment.
>
> On one tasktracker i stopped the network. On the jobtracker i could see the "Seconds since heartbeat"
> increasing. But after 60 seconds the tasktracker was still in the overview.
> Even in the log of the jobtracker I found nothing.
>
> After over 600 seconds i found the message
> org.apache.hadoop.mapred.JobTracker: Lost tracker .....
> And the tasktracker wasn't shown any more on the jobtracker.
>
> Isn't this the right setting?
>
> Regards Hansi
>
Aw: mapreduce.jobtracker.expire.trackers.interval no effect
Posted by Hansi Klose <ha...@web.de>.
Hi.
i think i found the reason. I looked at the job.xml and found the parameter
mapred.tasktracker.expiry.interval 6000000
and
mapreduce.jobtracker.expire.trackers.interval 30000
So i tried the deprecated parameter mapred.tasktracker.expiry.interval in my configuration and voila it works!
Why they write that the parameter is deprecated when the new one is not working and will be overwritten by the old one
with the default value?
http://archive.cloudera.com/cdh4/cdh/4/hadoop-2.0.0-cdh4.2.0/hadoop-project-dist/hadoop-common/DeprecatedProperties.html
Regards Hansi
> Gesendet: Dienstag, 03. Dezember 2013 um 17:38 Uhr
> Von: "Hansi Klose" <ha...@web.de>
> An: user@hadoop.apache.org
> Betreff: mapreduce.jobtracker.expire.trackers.interval no effect
>
> Hi,
>
> we want to set the heartbeat timout for a tasktracker.
>
> If the tasktracker does not send heartbeats for 60 seconds he should
> be marked as lost.
>
> I found the parameter mapreduce.jobtracker.expire.trackers.interval
> which sounds right to me.
>
> I set
>
> <property>
> <name>mapreduce.jobtracker.expire.trackers.interval</name>
> <value>60000</value>
> </property>
>
> in the mapred-site.xml on all servers and restarted the jobtracker and all tasktrackers.
>
> I started a benchmark "hadoop jar hadoop-examples.jar randomwriter rand" and every tasktracker gets 2 jobs.
> It is a small test environment.
>
> On one tasktracker i stopped the network. On the jobtracker i could see the "Seconds since heartbeat"
> increasing. But after 60 seconds the tasktracker was still in the overview.
> Even in the log of the jobtracker I found nothing.
>
> After over 600 seconds i found the message
> org.apache.hadoop.mapred.JobTracker: Lost tracker .....
> And the tasktracker wasn't shown any more on the jobtracker.
>
> Isn't this the right setting?
>
> Regards Hansi
>
Aw: mapreduce.jobtracker.expire.trackers.interval no effect
Posted by Hansi Klose <ha...@web.de>.
I forget to say that we use Cloudera 2.0.0-mr1-cdh4.2.0
> Gesendet: Dienstag, 03. Dezember 2013 um 17:38 Uhr
> Von: "Hansi Klose" <ha...@web.de>
> An: user@hadoop.apache.org
> Betreff: mapreduce.jobtracker.expire.trackers.interval no effect
>
> Hi,
>
> we want to set the heartbeat timout for a tasktracker.
>
> If the tasktracker does not send heartbeats for 60 seconds he should
> be marked as lost.
>
> I found the parameter mapreduce.jobtracker.expire.trackers.interval
> which sounds right to me.
>
> I set
>
> <property>
> <name>mapreduce.jobtracker.expire.trackers.interval</name>
> <value>60000</value>
> </property>
>
> in the mapred-site.xml on all servers and restarted the jobtracker and all tasktrackers.
>
> I started a benchmark "hadoop jar hadoop-examples.jar randomwriter rand" and every tasktracker gets 2 jobs.
> It is a small test environment.
>
> On one tasktracker i stopped the network. On the jobtracker i could see the "Seconds since heartbeat"
> increasing. But after 60 seconds the tasktracker was still in the overview.
> Even in the log of the jobtracker I found nothing.
>
> After over 600 seconds i found the message
> org.apache.hadoop.mapred.JobTracker: Lost tracker .....
> And the tasktracker wasn't shown any more on the jobtracker.
>
> Isn't this the right setting?
>
> Regards Hansi
>
Aw: mapreduce.jobtracker.expire.trackers.interval no effect
Posted by Hansi Klose <ha...@web.de>.
Hi.
i think i found the reason. I looked at the job.xml and found the parameter
mapred.tasktracker.expiry.interval 6000000
and
mapreduce.jobtracker.expire.trackers.interval 30000
So i tried the deprecated parameter mapred.tasktracker.expiry.interval in my configuration and voila it works!
Why they write that the parameter is deprecated when the new one is not working and will be overwritten by the old one
with the default value?
http://archive.cloudera.com/cdh4/cdh/4/hadoop-2.0.0-cdh4.2.0/hadoop-project-dist/hadoop-common/DeprecatedProperties.html
Regards Hansi
> Gesendet: Dienstag, 03. Dezember 2013 um 17:38 Uhr
> Von: "Hansi Klose" <ha...@web.de>
> An: user@hadoop.apache.org
> Betreff: mapreduce.jobtracker.expire.trackers.interval no effect
>
> Hi,
>
> we want to set the heartbeat timout for a tasktracker.
>
> If the tasktracker does not send heartbeats for 60 seconds he should
> be marked as lost.
>
> I found the parameter mapreduce.jobtracker.expire.trackers.interval
> which sounds right to me.
>
> I set
>
> <property>
> <name>mapreduce.jobtracker.expire.trackers.interval</name>
> <value>60000</value>
> </property>
>
> in the mapred-site.xml on all servers and restarted the jobtracker and all tasktrackers.
>
> I started a benchmark "hadoop jar hadoop-examples.jar randomwriter rand" and every tasktracker gets 2 jobs.
> It is a small test environment.
>
> On one tasktracker i stopped the network. On the jobtracker i could see the "Seconds since heartbeat"
> increasing. But after 60 seconds the tasktracker was still in the overview.
> Even in the log of the jobtracker I found nothing.
>
> After over 600 seconds i found the message
> org.apache.hadoop.mapred.JobTracker: Lost tracker .....
> And the tasktracker wasn't shown any more on the jobtracker.
>
> Isn't this the right setting?
>
> Regards Hansi
>