You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Brenden Matthews <br...@diddyinc.com> on 2013/06/03 20:02:20 UTC
Re: Review Request: Don't prematurely kill TaskTrackers.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11119/
-----------------------------------------------------------
(Updated June 3, 2013, 6:02 p.m.)
Review request for mesos.
Description (updated)
-------
Don't prematurely kill TaskTrackers.
We assign the JobID to the internal Mesos tracker (for the TaskTracker)
rather than the TaskID.
In the case where a TaskTracker has only map tasks assigned to it and
they have all completed, we mustn't terminate the tracker until the
entire job has finished, including the map tasks.
Review: https://reviews.apache.org/r/11119
Diffs (updated)
-----
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java afe401f5265e3d9494af7eace42eec45943184a3
Diff: https://reviews.apache.org/r/11119/diff/
Testing
-------
Used in production at airbnb.
Thanks,
Brenden Matthews
Re: Review Request: Don't prematurely kill TaskTrackers.
Posted by Vinod Kone <vi...@gmail.com>.
> On June 11, 2013, 10:03 p.m., Vinod Kone wrote:
> > hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java, lines 114-148
> > <https://reviews.apache.org/r/11119/diff/5/?file=303828#file303828line114>
> >
> > Are you missing a diff here? I don't see this resolved per my earlier comment?
>
> Brenden Matthews wrote:
> It's in later commits:
>
> https://reviews.apache.org/r/11603/
> https://reviews.apache.org/r/11489/
>
> I can drop the other request and roll it in to this one if you want. It's just very difficult for me to keep track of all the different changes.
Either that, or dicard this if those two reviews subsume these changes.
> On June 11, 2013, 10:03 p.m., Vinod Kone wrote:
> > hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java, lines 115-118
> > <https://reviews.apache.org/r/11119/diff/5/?file=303828#file303828line115>
> >
> > How is this possible?
>
> Brenden Matthews wrote:
> You can have different jobs on the same trackers.
I might be missing something, but you are looking at the job that is completed and the task list is from the completed job. Either way, looks like you dropped this in another review.
- Vinod
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11119/#review21749
-----------------------------------------------------------
On June 11, 2013, 10:12 p.m., Brenden Matthews wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11119/
> -----------------------------------------------------------
>
> (Updated June 11, 2013, 10:12 p.m.)
>
>
> Review request for mesos.
>
>
> Description
> -------
>
> Don't prematurely kill TaskTrackers.
>
> We assign the JobID to the internal Mesos tracker (for the TaskTracker)
> rather than the TaskID.
>
> In the case where a TaskTracker has only map tasks assigned to it and
> they have all completed, we mustn't terminate the tracker until the
> entire job has finished, including the map tasks.
>
> Review: https://reviews.apache.org/r/11119
>
>
> Diffs
> -----
>
> hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java afe401f5265e3d9494af7eace42eec45943184a3
>
> Diff: https://reviews.apache.org/r/11119/diff/
>
>
> Testing
> -------
>
> Used in production at airbnb.
>
> make -j10 check && cd hadoop && make hadoop-2.0.0-mr1-cdh4.2.1 && make hadoop-0.20.205.0 && make hadoop-0.20.2-cdh3u3
>
>
> Thanks,
>
> Brenden Matthews
>
>
Re: Review Request: Don't prematurely kill TaskTrackers.
Posted by Brenden Matthews <br...@diddyinc.com>.
> On June 11, 2013, 10:03 p.m., Vinod Kone wrote:
> > hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java, lines 114-148
> > <https://reviews.apache.org/r/11119/diff/5/?file=303828#file303828line114>
> >
> > Are you missing a diff here? I don't see this resolved per my earlier comment?
It's in later commits:
https://reviews.apache.org/r/11603/
https://reviews.apache.org/r/11489/
I can drop the other request and roll it in to this one if you want. It's just very difficult for me to keep track of all the different changes.
> On June 11, 2013, 10:03 p.m., Vinod Kone wrote:
> > hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java, lines 115-118
> > <https://reviews.apache.org/r/11119/diff/5/?file=303828#file303828line115>
> >
> > How is this possible?
You can have different jobs on the same trackers.
- Brenden
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11119/#review21749
-----------------------------------------------------------
On June 11, 2013, 9:42 p.m., Brenden Matthews wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11119/
> -----------------------------------------------------------
>
> (Updated June 11, 2013, 9:42 p.m.)
>
>
> Review request for mesos.
>
>
> Description
> -------
>
> Don't prematurely kill TaskTrackers.
>
> We assign the JobID to the internal Mesos tracker (for the TaskTracker)
> rather than the TaskID.
>
> In the case where a TaskTracker has only map tasks assigned to it and
> they have all completed, we mustn't terminate the tracker until the
> entire job has finished, including the map tasks.
>
> Review: https://reviews.apache.org/r/11119
>
>
> Diffs
> -----
>
> hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java afe401f5265e3d9494af7eace42eec45943184a3
>
> Diff: https://reviews.apache.org/r/11119/diff/
>
>
> Testing
> -------
>
> Used in production at airbnb.
>
> make -j10 check && cd hadoop && make hadoop-2.0.0-mr1-cdh4.2.1 && make hadoop-0.20.205.0 && make hadoop-0.20.2-cdh3u3
>
>
> Thanks,
>
> Brenden Matthews
>
>
Re: Review Request: Don't prematurely kill TaskTrackers.
Posted by Brenden Matthews <br...@diddyinc.com>.
> On June 11, 2013, 10:03 p.m., Vinod Kone wrote:
> > hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java, lines 114-148
> > <https://reviews.apache.org/r/11119/diff/5/?file=303828#file303828line114>
> >
> > Are you missing a diff here? I don't see this resolved per my earlier comment?
>
> Brenden Matthews wrote:
> It's in later commits:
>
> https://reviews.apache.org/r/11603/
> https://reviews.apache.org/r/11489/
>
> I can drop the other request and roll it in to this one if you want. It's just very difficult for me to keep track of all the different changes.
>
> Vinod Kone wrote:
> Either that, or dicard this if those two reviews subsume these changes.
>
> Brenden Matthews wrote:
> What's the procedure for capturing dependencies with review board? Most of my patches depend on previous ones. They need to be added sequentially. Here's the current state:
>
> https://github.com/airbnb/mesos/commits/testing
>
> Vinod Kone wrote:
> Unfortunately, I don't think there is a good way to capture dependencies. We have plans to augment our postreviews.py script to add some metadata (e.g., parent review id) in the commit message.
>
> For this particular one, I suggested to drop this only if other reviews subsumed the changes. Ofcourse, if there are some changes that are only in this patch, you cannot discard this.
I this case I think it's better to keep these as separate changes because there are several other patches between them. I want to a) keep the change history and b) avoid a merge nightmare.
Another problem is that this patch depends on others, in particular, this, which does not have a shipit:
https://reviews.apache.org/r/11118/
Can you take a look?
- Brenden
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11119/#review21749
-----------------------------------------------------------
On June 11, 2013, 10:12 p.m., Brenden Matthews wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11119/
> -----------------------------------------------------------
>
> (Updated June 11, 2013, 10:12 p.m.)
>
>
> Review request for mesos.
>
>
> Description
> -------
>
> Don't prematurely kill TaskTrackers.
>
> We assign the JobID to the internal Mesos tracker (for the TaskTracker)
> rather than the TaskID.
>
> In the case where a TaskTracker has only map tasks assigned to it and
> they have all completed, we mustn't terminate the tracker until the
> entire job has finished, including the map tasks.
>
> Review: https://reviews.apache.org/r/11119
>
>
> Diffs
> -----
>
> hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java afe401f5265e3d9494af7eace42eec45943184a3
>
> Diff: https://reviews.apache.org/r/11119/diff/
>
>
> Testing
> -------
>
> Used in production at airbnb.
>
> make -j10 check && cd hadoop && make hadoop-2.0.0-mr1-cdh4.2.1 && make hadoop-0.20.205.0 && make hadoop-0.20.2-cdh3u3
>
>
> Thanks,
>
> Brenden Matthews
>
>
Re: Review Request: Don't prematurely kill TaskTrackers.
Posted by Brenden Matthews <br...@diddyinc.com>.
> On June 11, 2013, 10:03 p.m., Vinod Kone wrote:
> > hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java, lines 114-148
> > <https://reviews.apache.org/r/11119/diff/5/?file=303828#file303828line114>
> >
> > Are you missing a diff here? I don't see this resolved per my earlier comment?
>
> Brenden Matthews wrote:
> It's in later commits:
>
> https://reviews.apache.org/r/11603/
> https://reviews.apache.org/r/11489/
>
> I can drop the other request and roll it in to this one if you want. It's just very difficult for me to keep track of all the different changes.
>
> Vinod Kone wrote:
> Either that, or dicard this if those two reviews subsume these changes.
What's the procedure for capturing dependencies with review board? Most of my patches depend on previous ones. They need to be added sequentially. Here's the current state:
https://github.com/airbnb/mesos/commits/testing
- Brenden
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11119/#review21749
-----------------------------------------------------------
On June 11, 2013, 10:12 p.m., Brenden Matthews wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11119/
> -----------------------------------------------------------
>
> (Updated June 11, 2013, 10:12 p.m.)
>
>
> Review request for mesos.
>
>
> Description
> -------
>
> Don't prematurely kill TaskTrackers.
>
> We assign the JobID to the internal Mesos tracker (for the TaskTracker)
> rather than the TaskID.
>
> In the case where a TaskTracker has only map tasks assigned to it and
> they have all completed, we mustn't terminate the tracker until the
> entire job has finished, including the map tasks.
>
> Review: https://reviews.apache.org/r/11119
>
>
> Diffs
> -----
>
> hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java afe401f5265e3d9494af7eace42eec45943184a3
>
> Diff: https://reviews.apache.org/r/11119/diff/
>
>
> Testing
> -------
>
> Used in production at airbnb.
>
> make -j10 check && cd hadoop && make hadoop-2.0.0-mr1-cdh4.2.1 && make hadoop-0.20.205.0 && make hadoop-0.20.2-cdh3u3
>
>
> Thanks,
>
> Brenden Matthews
>
>
Re: Review Request: Don't prematurely kill TaskTrackers.
Posted by Vinod Kone <vi...@gmail.com>.
> On June 11, 2013, 10:03 p.m., Vinod Kone wrote:
> > hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java, lines 114-148
> > <https://reviews.apache.org/r/11119/diff/5/?file=303828#file303828line114>
> >
> > Are you missing a diff here? I don't see this resolved per my earlier comment?
>
> Brenden Matthews wrote:
> It's in later commits:
>
> https://reviews.apache.org/r/11603/
> https://reviews.apache.org/r/11489/
>
> I can drop the other request and roll it in to this one if you want. It's just very difficult for me to keep track of all the different changes.
>
> Vinod Kone wrote:
> Either that, or dicard this if those two reviews subsume these changes.
>
> Brenden Matthews wrote:
> What's the procedure for capturing dependencies with review board? Most of my patches depend on previous ones. They need to be added sequentially. Here's the current state:
>
> https://github.com/airbnb/mesos/commits/testing
Unfortunately, I don't think there is a good way to capture dependencies. We have plans to augment our postreviews.py script to add some metadata (e.g., parent review id) in the commit message.
For this particular one, I suggested to drop this only if other reviews subsumed the changes. Ofcourse, if there are some changes that are only in this patch, you cannot discard this.
- Vinod
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11119/#review21749
-----------------------------------------------------------
On June 11, 2013, 10:12 p.m., Brenden Matthews wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11119/
> -----------------------------------------------------------
>
> (Updated June 11, 2013, 10:12 p.m.)
>
>
> Review request for mesos.
>
>
> Description
> -------
>
> Don't prematurely kill TaskTrackers.
>
> We assign the JobID to the internal Mesos tracker (for the TaskTracker)
> rather than the TaskID.
>
> In the case where a TaskTracker has only map tasks assigned to it and
> they have all completed, we mustn't terminate the tracker until the
> entire job has finished, including the map tasks.
>
> Review: https://reviews.apache.org/r/11119
>
>
> Diffs
> -----
>
> hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java afe401f5265e3d9494af7eace42eec45943184a3
>
> Diff: https://reviews.apache.org/r/11119/diff/
>
>
> Testing
> -------
>
> Used in production at airbnb.
>
> make -j10 check && cd hadoop && make hadoop-2.0.0-mr1-cdh4.2.1 && make hadoop-0.20.205.0 && make hadoop-0.20.2-cdh3u3
>
>
> Thanks,
>
> Brenden Matthews
>
>
Re: Review Request: Don't prematurely kill TaskTrackers.
Posted by Vinod Kone <vi...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11119/#review21749
-----------------------------------------------------------
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java
<https://reviews.apache.org/r/11119/#comment44939>
Are you missing a diff here? I don't see this resolved per my earlier comment?
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java
<https://reviews.apache.org/r/11119/#comment44938>
How is this possible?
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java
<https://reviews.apache.org/r/11119/#comment44937>
This is going to print the same job id multiple times as we add tasks of the same job. Can you revert this?
- Vinod Kone
On June 11, 2013, 9:42 p.m., Brenden Matthews wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11119/
> -----------------------------------------------------------
>
> (Updated June 11, 2013, 9:42 p.m.)
>
>
> Review request for mesos.
>
>
> Description
> -------
>
> Don't prematurely kill TaskTrackers.
>
> We assign the JobID to the internal Mesos tracker (for the TaskTracker)
> rather than the TaskID.
>
> In the case where a TaskTracker has only map tasks assigned to it and
> they have all completed, we mustn't terminate the tracker until the
> entire job has finished, including the map tasks.
>
> Review: https://reviews.apache.org/r/11119
>
>
> Diffs
> -----
>
> hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java afe401f5265e3d9494af7eace42eec45943184a3
>
> Diff: https://reviews.apache.org/r/11119/diff/
>
>
> Testing
> -------
>
> Used in production at airbnb.
>
> make -j10 check && cd hadoop && make hadoop-2.0.0-mr1-cdh4.2.1 && make hadoop-0.20.205.0 && make hadoop-0.20.2-cdh3u3
>
>
> Thanks,
>
> Brenden Matthews
>
>
Re: Review Request 11119: Don't prematurely kill TaskTrackers.
Posted by Vinod Kone <vi...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11119/#review23799
-----------------------------------------------------------
Ship it!
Ship It!
- Vinod Kone
On July 24, 2013, 10:19 p.m., Brenden Matthews wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11119/
> -----------------------------------------------------------
>
> (Updated July 24, 2013, 10:19 p.m.)
>
>
> Review request for mesos.
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Don't prematurely kill TaskTrackers.
>
> We assign the JobID to the internal Mesos tracker (for the TaskTracker)
> rather than the TaskID.
>
> In the case where a TaskTracker has only map tasks assigned to it and
> they have all completed, we mustn't terminate the tracker until the
> entire job has finished, including the map tasks.
>
> Review: https://reviews.apache.org/r/11119
>
>
> Diffs
> -----
>
> hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java 279f84e0f0c43ad3cfd9e4442010e706ee3565d9
>
> Diff: https://reviews.apache.org/r/11119/diff/
>
>
> Testing
> -------
>
> Used in production at airbnb.
>
> make -j10 check && cd hadoop && make hadoop-2.0.0-mr1-cdh4.2.1 && make hadoop-0.20.205.0 && make hadoop-0.20.2-cdh3u3
>
>
> Thanks,
>
> Brenden Matthews
>
>
Re: Review Request 11119: Don't prematurely kill TaskTrackers.
Posted by Brenden Matthews <br...@diddyinc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11119/
-----------------------------------------------------------
(Updated July 30, 2013, 7:35 p.m.)
Review request for mesos.
Changes
-------
Fix dependency.
Repository: mesos
Description
-------
Don't prematurely kill TaskTrackers.
We assign the JobID to the internal Mesos tracker (for the TaskTracker)
rather than the TaskID.
In the case where a TaskTracker has only map tasks assigned to it and
they have all completed, we mustn't terminate the tracker until the
entire job has finished, including the map tasks.
Review: https://reviews.apache.org/r/11119
Diffs
-----
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java 279f84e0f0c43ad3cfd9e4442010e706ee3565d9
Diff: https://reviews.apache.org/r/11119/diff/
Testing
-------
Used in production at airbnb.
make -j10 check && cd hadoop && make hadoop-2.0.0-mr1-cdh4.2.1 && make hadoop-0.20.205.0 && make hadoop-0.20.2-cdh3u3
Thanks,
Brenden Matthews
Re: Review Request 11119: Don't prematurely kill TaskTrackers.
Posted by Brenden Matthews <br...@diddyinc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11119/
-----------------------------------------------------------
(Updated July 30, 2013, 7:27 p.m.)
Review request for mesos.
Changes
-------
Rebasing on master, updating dependencies.
Repository: mesos
Description
-------
Don't prematurely kill TaskTrackers.
We assign the JobID to the internal Mesos tracker (for the TaskTracker)
rather than the TaskID.
In the case where a TaskTracker has only map tasks assigned to it and
they have all completed, we mustn't terminate the tracker until the
entire job has finished, including the map tasks.
Review: https://reviews.apache.org/r/11119
Diffs
-----
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java 279f84e0f0c43ad3cfd9e4442010e706ee3565d9
Diff: https://reviews.apache.org/r/11119/diff/
Testing
-------
Used in production at airbnb.
make -j10 check && cd hadoop && make hadoop-2.0.0-mr1-cdh4.2.1 && make hadoop-0.20.205.0 && make hadoop-0.20.2-cdh3u3
Thanks,
Brenden Matthews
Re: Review Request 11119: Don't prematurely kill TaskTrackers.
Posted by Brenden Matthews <br...@diddyinc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11119/
-----------------------------------------------------------
(Updated July 25, 2013, 1:29 a.m.)
Review request for mesos.
Changes
-------
Rebase on master.
Repository: mesos
Description
-------
Don't prematurely kill TaskTrackers.
We assign the JobID to the internal Mesos tracker (for the TaskTracker)
rather than the TaskID.
In the case where a TaskTracker has only map tasks assigned to it and
they have all completed, we mustn't terminate the tracker until the
entire job has finished, including the map tasks.
Review: https://reviews.apache.org/r/11119
Diffs (updated)
-----
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java 279f84e0f0c43ad3cfd9e4442010e706ee3565d9
Diff: https://reviews.apache.org/r/11119/diff/
Testing
-------
Used in production at airbnb.
make -j10 check && cd hadoop && make hadoop-2.0.0-mr1-cdh4.2.1 && make hadoop-0.20.205.0 && make hadoop-0.20.2-cdh3u3
Thanks,
Brenden Matthews
Re: Review Request 11119: Don't prematurely kill TaskTrackers.
Posted by Brenden Matthews <br...@diddyinc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11119/
-----------------------------------------------------------
(Updated July 24, 2013, 10:19 p.m.)
Review request for mesos.
Changes
-------
Rebasing on master.
Repository: mesos
Description
-------
Don't prematurely kill TaskTrackers.
We assign the JobID to the internal Mesos tracker (for the TaskTracker)
rather than the TaskID.
In the case where a TaskTracker has only map tasks assigned to it and
they have all completed, we mustn't terminate the tracker until the
entire job has finished, including the map tasks.
Review: https://reviews.apache.org/r/11119
Diffs (updated)
-----
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java 279f84e0f0c43ad3cfd9e4442010e706ee3565d9
Diff: https://reviews.apache.org/r/11119/diff/
Testing
-------
Used in production at airbnb.
make -j10 check && cd hadoop && make hadoop-2.0.0-mr1-cdh4.2.1 && make hadoop-0.20.205.0 && make hadoop-0.20.2-cdh3u3
Thanks,
Brenden Matthews
Re: Review Request: Don't prematurely kill TaskTrackers.
Posted by Brenden Matthews <br...@diddyinc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11119/
-----------------------------------------------------------
(Updated June 11, 2013, 10:12 p.m.)
Review request for mesos.
Description
-------
Don't prematurely kill TaskTrackers.
We assign the JobID to the internal Mesos tracker (for the TaskTracker)
rather than the TaskID.
In the case where a TaskTracker has only map tasks assigned to it and
they have all completed, we mustn't terminate the tracker until the
entire job has finished, including the map tasks.
Review: https://reviews.apache.org/r/11119
Diffs (updated)
-----
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java afe401f5265e3d9494af7eace42eec45943184a3
Diff: https://reviews.apache.org/r/11119/diff/
Testing
-------
Used in production at airbnb.
make -j10 check && cd hadoop && make hadoop-2.0.0-mr1-cdh4.2.1 && make hadoop-0.20.205.0 && make hadoop-0.20.2-cdh3u3
Thanks,
Brenden Matthews
Re: Review Request: Don't prematurely kill TaskTrackers.
Posted by Brenden Matthews <br...@diddyinc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11119/
-----------------------------------------------------------
(Updated June 11, 2013, 9:42 p.m.)
Review request for mesos.
Changes
-------
Rebasing on master.
Description
-------
Don't prematurely kill TaskTrackers.
We assign the JobID to the internal Mesos tracker (for the TaskTracker)
rather than the TaskID.
In the case where a TaskTracker has only map tasks assigned to it and
they have all completed, we mustn't terminate the tracker until the
entire job has finished, including the map tasks.
Review: https://reviews.apache.org/r/11119
Diffs (updated)
-----
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java afe401f5265e3d9494af7eace42eec45943184a3
Diff: https://reviews.apache.org/r/11119/diff/
Testing (updated)
-------
Used in production at airbnb.
make -j10 check && cd hadoop && make hadoop-2.0.0-mr1-cdh4.2.1 && make hadoop-0.20.205.0 && make hadoop-0.20.2-cdh3u3
Thanks,
Brenden Matthews
Re: Review Request: Don't prematurely kill TaskTrackers.
Posted by Brenden Matthews <br...@diddyinc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11119/
-----------------------------------------------------------
(Updated June 6, 2013, 2:11 a.m.)
Review request for mesos.
Changes
-------
Rebasing on master.
Description
-------
Don't prematurely kill TaskTrackers.
We assign the JobID to the internal Mesos tracker (for the TaskTracker)
rather than the TaskID.
In the case where a TaskTracker has only map tasks assigned to it and
they have all completed, we mustn't terminate the tracker until the
entire job has finished, including the map tasks.
Review: https://reviews.apache.org/r/11119
Diffs (updated)
-----
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java afe401f5265e3d9494af7eace42eec45943184a3
Diff: https://reviews.apache.org/r/11119/diff/
Testing
-------
Used in production at airbnb.
Thanks,
Brenden Matthews