You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Brenden Matthews <br...@diddyinc.com> on 2013/07/25 00:19:16 UTC

Re: Review Request 11119: Don't prematurely kill TaskTrackers.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11119/
-----------------------------------------------------------

(Updated July 24, 2013, 10:19 p.m.)


Review request for mesos.


Changes
-------

Rebasing on master.


Repository: mesos


Description
-------

Don't prematurely kill TaskTrackers.

We assign the JobID to the internal Mesos tracker (for the TaskTracker)
rather than the TaskID.

In the case where a TaskTracker has only map tasks assigned to it and
they have all completed, we mustn't terminate the tracker until the
entire job has finished, including the map tasks.

Review: https://reviews.apache.org/r/11119


Diffs (updated)
-----

  hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java 279f84e0f0c43ad3cfd9e4442010e706ee3565d9 

Diff: https://reviews.apache.org/r/11119/diff/


Testing
-------

Used in production at airbnb.

make -j10 check && cd hadoop && make hadoop-2.0.0-mr1-cdh4.2.1 && make hadoop-0.20.205.0 && make hadoop-0.20.2-cdh3u3


Thanks,

Brenden Matthews


Re: Review Request 11119: Don't prematurely kill TaskTrackers.

Posted by Vinod Kone <vi...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11119/#review23799
-----------------------------------------------------------

Ship it!


Ship It!

- Vinod Kone


On July 24, 2013, 10:19 p.m., Brenden Matthews wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11119/
> -----------------------------------------------------------
> 
> (Updated July 24, 2013, 10:19 p.m.)
> 
> 
> Review request for mesos.
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Don't prematurely kill TaskTrackers.
> 
> We assign the JobID to the internal Mesos tracker (for the TaskTracker)
> rather than the TaskID.
> 
> In the case where a TaskTracker has only map tasks assigned to it and
> they have all completed, we mustn't terminate the tracker until the
> entire job has finished, including the map tasks.
> 
> Review: https://reviews.apache.org/r/11119
> 
> 
> Diffs
> -----
> 
>   hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java 279f84e0f0c43ad3cfd9e4442010e706ee3565d9 
> 
> Diff: https://reviews.apache.org/r/11119/diff/
> 
> 
> Testing
> -------
> 
> Used in production at airbnb.
> 
> make -j10 check && cd hadoop && make hadoop-2.0.0-mr1-cdh4.2.1 && make hadoop-0.20.205.0 && make hadoop-0.20.2-cdh3u3
> 
> 
> Thanks,
> 
> Brenden Matthews
> 
>


Re: Review Request 11119: Don't prematurely kill TaskTrackers.

Posted by Brenden Matthews <br...@diddyinc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11119/
-----------------------------------------------------------

(Updated July 30, 2013, 7:35 p.m.)


Review request for mesos.


Changes
-------

Fix dependency.


Repository: mesos


Description
-------

Don't prematurely kill TaskTrackers.

We assign the JobID to the internal Mesos tracker (for the TaskTracker)
rather than the TaskID.

In the case where a TaskTracker has only map tasks assigned to it and
they have all completed, we mustn't terminate the tracker until the
entire job has finished, including the map tasks.

Review: https://reviews.apache.org/r/11119


Diffs
-----

  hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java 279f84e0f0c43ad3cfd9e4442010e706ee3565d9 

Diff: https://reviews.apache.org/r/11119/diff/


Testing
-------

Used in production at airbnb.

make -j10 check && cd hadoop && make hadoop-2.0.0-mr1-cdh4.2.1 && make hadoop-0.20.205.0 && make hadoop-0.20.2-cdh3u3


Thanks,

Brenden Matthews


Re: Review Request 11119: Don't prematurely kill TaskTrackers.

Posted by Brenden Matthews <br...@diddyinc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11119/
-----------------------------------------------------------

(Updated July 30, 2013, 7:27 p.m.)


Review request for mesos.


Changes
-------

Rebasing on master, updating dependencies.


Repository: mesos


Description
-------

Don't prematurely kill TaskTrackers.

We assign the JobID to the internal Mesos tracker (for the TaskTracker)
rather than the TaskID.

In the case where a TaskTracker has only map tasks assigned to it and
they have all completed, we mustn't terminate the tracker until the
entire job has finished, including the map tasks.

Review: https://reviews.apache.org/r/11119


Diffs
-----

  hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java 279f84e0f0c43ad3cfd9e4442010e706ee3565d9 

Diff: https://reviews.apache.org/r/11119/diff/


Testing
-------

Used in production at airbnb.

make -j10 check && cd hadoop && make hadoop-2.0.0-mr1-cdh4.2.1 && make hadoop-0.20.205.0 && make hadoop-0.20.2-cdh3u3


Thanks,

Brenden Matthews


Re: Review Request 11119: Don't prematurely kill TaskTrackers.

Posted by Brenden Matthews <br...@diddyinc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11119/
-----------------------------------------------------------

(Updated July 25, 2013, 1:29 a.m.)


Review request for mesos.


Changes
-------

Rebase on master.


Repository: mesos


Description
-------

Don't prematurely kill TaskTrackers.

We assign the JobID to the internal Mesos tracker (for the TaskTracker)
rather than the TaskID.

In the case where a TaskTracker has only map tasks assigned to it and
they have all completed, we mustn't terminate the tracker until the
entire job has finished, including the map tasks.

Review: https://reviews.apache.org/r/11119


Diffs (updated)
-----

  hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java 279f84e0f0c43ad3cfd9e4442010e706ee3565d9 

Diff: https://reviews.apache.org/r/11119/diff/


Testing
-------

Used in production at airbnb.

make -j10 check && cd hadoop && make hadoop-2.0.0-mr1-cdh4.2.1 && make hadoop-0.20.205.0 && make hadoop-0.20.2-cdh3u3


Thanks,

Brenden Matthews