You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Brenden Matthews <br...@diddyinc.com> on 2013/05/14 02:19:25 UTC
Review Request: Run Hadoop tutorial binaries from within build dir.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11109/
-----------------------------------------------------------
Review request for mesos.
Description
-------
>From e38084684887f59bce5694bb3457a94b2da026e3 Mon Sep 17 00:00:00 2001
From: Brenden Matthews <br...@airbnb.com>
Date: Mon, 15 Apr 2013 14:08:01 -0700
Subject: [PATCH 02/24] Run Hadoop tutorial binaries from within build dir.
Review: https://reviews.apache.org/r/11109
---
hadoop/TUTORIAL.sh | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
Diffs
-----
hadoop/TUTORIAL.sh d81c55562d217e8784745d4a8cedf78ada9f5453
Diff: https://reviews.apache.org/r/11109/diff/
Testing
-------
Used in production at airbnb.
Thanks,
Brenden Matthews
Re: Review Request: Run Hadoop tutorial binaries from within build dir.
Posted by Vinod Kone <vi...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11109/#review21510
-----------------------------------------------------------
hadoop/TUTORIAL.sh
<https://reviews.apache.org/r/11109/#comment44550>
This comment no longer makes sense.
hadoop/TUTORIAL.sh
<https://reviews.apache.org/r/11109/#comment44551>
how about
mv ${hadoop} ${hadoop}-mesos ?
hadoop/TUTORIAL.sh
<https://reviews.apache.org/r/11109/#comment44552>
s/hadoop-{distribution}/{hadoop}/
hadoop/TUTORIAL.sh
<https://reviews.apache.org/r/11109/#comment44553>
ditto
hadoop/TUTORIAL.sh
<https://reviews.apache.org/r/11109/#comment44554>
ditto
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java
<https://reviews.apache.org/r/11109/#comment44555>
nice trick!
- Vinod Kone
On June 3, 2013, 5:53 p.m., Brenden Matthews wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11109/
> -----------------------------------------------------------
>
> (Updated June 3, 2013, 5:53 p.m.)
>
>
> Review request for mesos.
>
>
> Description
> -------
>
> Run Hadoop tutorial binaries from within build dir.
>
> Review: https://reviews.apache.org/r/11109
>
>
> Diffs
> -----
>
> hadoop/TUTORIAL.sh d81c55562d217e8784745d4a8cedf78ada9f5453
> hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java afe401f5265e3d9494af7eace42eec45943184a3
>
> Diff: https://reviews.apache.org/r/11109/diff/
>
>
> Testing
> -------
>
> Used in production at airbnb.
>
>
> Thanks,
>
> Brenden Matthews
>
>
Re: Review Request: Run Hadoop tutorial binaries from within build dir.
Posted by Ben Mahler <be...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11109/#review21369
-----------------------------------------------------------
Ship it!
Ship It!
- Ben Mahler
On June 3, 2013, 5:53 p.m., Brenden Matthews wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11109/
> -----------------------------------------------------------
>
> (Updated June 3, 2013, 5:53 p.m.)
>
>
> Review request for mesos.
>
>
> Description
> -------
>
> Run Hadoop tutorial binaries from within build dir.
>
> Review: https://reviews.apache.org/r/11109
>
>
> Diffs
> -----
>
> hadoop/TUTORIAL.sh d81c55562d217e8784745d4a8cedf78ada9f5453
> hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java afe401f5265e3d9494af7eace42eec45943184a3
>
> Diff: https://reviews.apache.org/r/11109/diff/
>
>
> Testing
> -------
>
> Used in production at airbnb.
>
>
> Thanks,
>
> Brenden Matthews
>
>
Re: Review Request: Run Hadoop tutorial binaries from within build dir.
Posted by Brenden Matthews <br...@diddyinc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11109/
-----------------------------------------------------------
(Updated June 3, 2013, 5:53 p.m.)
Review request for mesos.
Description (updated)
-------
Run Hadoop tutorial binaries from within build dir.
Review: https://reviews.apache.org/r/11109
Diffs (updated)
-----
hadoop/TUTORIAL.sh d81c55562d217e8784745d4a8cedf78ada9f5453
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java afe401f5265e3d9494af7eace42eec45943184a3
Diff: https://reviews.apache.org/r/11109/diff/
Testing
-------
Used in production at airbnb.
Thanks,
Brenden Matthews
Re: Review Request: Run Hadoop tutorial binaries from within build dir.
Posted by Brenden Matthews <br...@diddyinc.com>.
> On June 3, 2013, 5:49 p.m., Ben Mahler wrote:
> > hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java, line 496
> > <https://reviews.apache.org/r/11109/diff/5/?file=295588#file295588line496>
> >
> > "cd hadoop*" is unfortunate, but I don't see a simple way to know the exact path, outside of patching the file.
> >
> > Also, s/cd hadoop*/cd hadoop-*/
It's how the Storm Mesos scheduler does it, so I decided to follow their lead.
- Brenden
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11109/#review21330
-----------------------------------------------------------
On May 22, 2013, 9:51 p.m., Brenden Matthews wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11109/
> -----------------------------------------------------------
>
> (Updated May 22, 2013, 9:51 p.m.)
>
>
> Review request for mesos.
>
>
> Description
> -------
>
> From e38084684887f59bce5694bb3457a94b2da026e3 Mon Sep 17 00:00:00 2001
> From: Brenden Matthews <br...@airbnb.com>
> Date: Mon, 15 Apr 2013 14:08:01 -0700
> Subject: [PATCH 02/24] Run Hadoop tutorial binaries from within build dir.
>
> Review: https://reviews.apache.org/r/11109
> ---
> hadoop/TUTORIAL.sh | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
>
> Diffs
> -----
>
> hadoop/TUTORIAL.sh d81c55562d217e8784745d4a8cedf78ada9f5453
> hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java afe401f5265e3d9494af7eace42eec45943184a3
>
> Diff: https://reviews.apache.org/r/11109/diff/
>
>
> Testing
> -------
>
> Used in production at airbnb.
>
>
> Thanks,
>
> Brenden Matthews
>
>
Re: Review Request: Run Hadoop tutorial binaries from within build dir.
Posted by Ben Mahler <be...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11109/#review21330
-----------------------------------------------------------
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java
<https://reviews.apache.org/r/11109/#comment44238>
"cd hadoop*" is unfortunate, but I don't see a simple way to know the exact path, outside of patching the file.
Also, s/cd hadoop*/cd hadoop-*/
- Ben Mahler
On May 22, 2013, 9:51 p.m., Brenden Matthews wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11109/
> -----------------------------------------------------------
>
> (Updated May 22, 2013, 9:51 p.m.)
>
>
> Review request for mesos.
>
>
> Description
> -------
>
> From e38084684887f59bce5694bb3457a94b2da026e3 Mon Sep 17 00:00:00 2001
> From: Brenden Matthews <br...@airbnb.com>
> Date: Mon, 15 Apr 2013 14:08:01 -0700
> Subject: [PATCH 02/24] Run Hadoop tutorial binaries from within build dir.
>
> Review: https://reviews.apache.org/r/11109
> ---
> hadoop/TUTORIAL.sh | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
>
> Diffs
> -----
>
> hadoop/TUTORIAL.sh d81c55562d217e8784745d4a8cedf78ada9f5453
> hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java afe401f5265e3d9494af7eace42eec45943184a3
>
> Diff: https://reviews.apache.org/r/11109/diff/
>
>
> Testing
> -------
>
> Used in production at airbnb.
>
>
> Thanks,
>
> Brenden Matthews
>
>
Re: Review Request: Run Hadoop tutorial binaries from within build dir.
Posted by Brenden Matthews <br...@diddyinc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11109/
-----------------------------------------------------------
(Updated May 22, 2013, 9:51 p.m.)
Review request for mesos.
Description
-------
>From e38084684887f59bce5694bb3457a94b2da026e3 Mon Sep 17 00:00:00 2001
From: Brenden Matthews <br...@airbnb.com>
Date: Mon, 15 Apr 2013 14:08:01 -0700
Subject: [PATCH 02/24] Run Hadoop tutorial binaries from within build dir.
Review: https://reviews.apache.org/r/11109
---
hadoop/TUTORIAL.sh | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
Diffs (updated)
-----
hadoop/TUTORIAL.sh d81c55562d217e8784745d4a8cedf78ada9f5453
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java afe401f5265e3d9494af7eace42eec45943184a3
Diff: https://reviews.apache.org/r/11109/diff/
Testing
-------
Used in production at airbnb.
Thanks,
Brenden Matthews
Re: Review Request: Run Hadoop tutorial binaries from within build dir.
Posted by Brenden Matthews <br...@diddyinc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11109/
-----------------------------------------------------------
(Updated May 22, 2013, 9:48 p.m.)
Review request for mesos.
Description
-------
>From e38084684887f59bce5694bb3457a94b2da026e3 Mon Sep 17 00:00:00 2001
From: Brenden Matthews <br...@airbnb.com>
Date: Mon, 15 Apr 2013 14:08:01 -0700
Subject: [PATCH 02/24] Run Hadoop tutorial binaries from within build dir.
Review: https://reviews.apache.org/r/11109
---
hadoop/TUTORIAL.sh | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
Diffs (updated)
-----
hadoop/TUTORIAL.sh d81c55562d217e8784745d4a8cedf78ada9f5453
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java afe401f5265e3d9494af7eace42eec45943184a3
support/post-reviews.py 4099d2c2d00ef2e8fac5369153329dd3de0998c1
Diff: https://reviews.apache.org/r/11109/diff/
Testing
-------
Used in production at airbnb.
Thanks,
Brenden Matthews
Re: Review Request: Run Hadoop tutorial binaries from within build dir.
Posted by Brenden Matthews <br...@diddyinc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11109/
-----------------------------------------------------------
(Updated May 22, 2013, 8:18 p.m.)
Review request for mesos.
Description
-------
>From e38084684887f59bce5694bb3457a94b2da026e3 Mon Sep 17 00:00:00 2001
From: Brenden Matthews <br...@airbnb.com>
Date: Mon, 15 Apr 2013 14:08:01 -0700
Subject: [PATCH 02/24] Run Hadoop tutorial binaries from within build dir.
Review: https://reviews.apache.org/r/11109
---
hadoop/TUTORIAL.sh | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
Diffs (updated)
-----
hadoop/TUTORIAL.sh d81c55562d217e8784745d4a8cedf78ada9f5453
Diff: https://reviews.apache.org/r/11109/diff/
Testing
-------
Used in production at airbnb.
Thanks,
Brenden Matthews
Re: Review Request: Run Hadoop tutorial binaries from within build dir.
Posted by Benjamin Hindman <be...@berkeley.edu>.
> On May 16, 2013, 9:45 p.m., Benjamin Hindman wrote:
> > hadoop/TUTORIAL.sh, line 578
> > <https://reviews.apache.org/r/11109/diff/2/?file=291854#file291854line578>
> >
> > So to be clear about my comment above, this should be:
> > mv ${hadoop} hadoop-${distribution}-mesos
Sorry, I meant to modify the 'tar' command.
- Benjamin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11109/#review20662
-----------------------------------------------------------
On May 14, 2013, 5:57 p.m., Brenden Matthews wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11109/
> -----------------------------------------------------------
>
> (Updated May 14, 2013, 5:57 p.m.)
>
>
> Review request for mesos.
>
>
> Description
> -------
>
> From e38084684887f59bce5694bb3457a94b2da026e3 Mon Sep 17 00:00:00 2001
> From: Brenden Matthews <br...@airbnb.com>
> Date: Mon, 15 Apr 2013 14:08:01 -0700
> Subject: [PATCH 02/24] Run Hadoop tutorial binaries from within build dir.
>
> Review: https://reviews.apache.org/r/11109
> ---
> hadoop/TUTORIAL.sh | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
>
> Diffs
> -----
>
> hadoop/TUTORIAL.sh d81c55562d217e8784745d4a8cedf78ada9f5453
>
> Diff: https://reviews.apache.org/r/11109/diff/
>
>
> Testing
> -------
>
> Used in production at airbnb.
>
>
> Thanks,
>
> Brenden Matthews
>
>
Re: Review Request: Run Hadoop tutorial binaries from within build dir.
Posted by Benjamin Hindman <be...@berkeley.edu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11109/#review20662
-----------------------------------------------------------
hadoop/TUTORIAL.sh
<https://reviews.apache.org/r/11109/#comment42659>
So to be clear about my comment above, this should be:
mv ${hadoop} hadoop-${distribution}-mesos
- Benjamin Hindman
On May 14, 2013, 5:57 p.m., Brenden Matthews wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11109/
> -----------------------------------------------------------
>
> (Updated May 14, 2013, 5:57 p.m.)
>
>
> Review request for mesos.
>
>
> Description
> -------
>
> From e38084684887f59bce5694bb3457a94b2da026e3 Mon Sep 17 00:00:00 2001
> From: Brenden Matthews <br...@airbnb.com>
> Date: Mon, 15 Apr 2013 14:08:01 -0700
> Subject: [PATCH 02/24] Run Hadoop tutorial binaries from within build dir.
>
> Review: https://reviews.apache.org/r/11109
> ---
> hadoop/TUTORIAL.sh | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
>
> Diffs
> -----
>
> hadoop/TUTORIAL.sh d81c55562d217e8784745d4a8cedf78ada9f5453
>
> Diff: https://reviews.apache.org/r/11109/diff/
>
>
> Testing
> -------
>
> Used in production at airbnb.
>
>
> Thanks,
>
> Brenden Matthews
>
>
Re: Review Request: Run Hadoop tutorial binaries from within build dir.
Posted by Benjamin Hindman <be...@berkeley.edu>.
> On May 16, 2013, 9:01 p.m., Vinod Kone wrote:
> > hadoop/TUTORIAL.sh, line 586
> > <https://reviews.apache.org/r/11109/diff/2/?file=291854#file291854line586>
> >
> > I think I agree that running hadoop out of the distribution is the right thing to do.
> >
> > But, can you add a comment (that gets printed out) here stating that why we are doing so?
Yes, I completely agree that running the example out of the "Mesos distribution" is the right thing to do. Thanks Brenden.
- Benjamin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11109/#review20656
-----------------------------------------------------------
On May 14, 2013, 5:57 p.m., Brenden Matthews wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11109/
> -----------------------------------------------------------
>
> (Updated May 14, 2013, 5:57 p.m.)
>
>
> Review request for mesos.
>
>
> Description
> -------
>
> From e38084684887f59bce5694bb3457a94b2da026e3 Mon Sep 17 00:00:00 2001
> From: Brenden Matthews <br...@airbnb.com>
> Date: Mon, 15 Apr 2013 14:08:01 -0700
> Subject: [PATCH 02/24] Run Hadoop tutorial binaries from within build dir.
>
> Review: https://reviews.apache.org/r/11109
> ---
> hadoop/TUTORIAL.sh | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
>
> Diffs
> -----
>
> hadoop/TUTORIAL.sh d81c55562d217e8784745d4a8cedf78ada9f5453
>
> Diff: https://reviews.apache.org/r/11109/diff/
>
>
> Testing
> -------
>
> Used in production at airbnb.
>
>
> Thanks,
>
> Brenden Matthews
>
>
Re: Review Request: Run Hadoop tutorial binaries from within build dir.
Posted by Vinod Kone <vi...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11109/#review20656
-----------------------------------------------------------
hadoop/TUTORIAL.sh
<https://reviews.apache.org/r/11109/#comment42649>
I think I agree that running hadoop out of the distribution is the right thing to do.
But, can you add a comment (that gets printed out) here stating that why we are doing so?
- Vinod Kone
On May 14, 2013, 5:57 p.m., Brenden Matthews wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11109/
> -----------------------------------------------------------
>
> (Updated May 14, 2013, 5:57 p.m.)
>
>
> Review request for mesos.
>
>
> Description
> -------
>
> From e38084684887f59bce5694bb3457a94b2da026e3 Mon Sep 17 00:00:00 2001
> From: Brenden Matthews <br...@airbnb.com>
> Date: Mon, 15 Apr 2013 14:08:01 -0700
> Subject: [PATCH 02/24] Run Hadoop tutorial binaries from within build dir.
>
> Review: https://reviews.apache.org/r/11109
> ---
> hadoop/TUTORIAL.sh | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
>
> Diffs
> -----
>
> hadoop/TUTORIAL.sh d81c55562d217e8784745d4a8cedf78ada9f5453
>
> Diff: https://reviews.apache.org/r/11109/diff/
>
>
> Testing
> -------
>
> Used in production at airbnb.
>
>
> Thanks,
>
> Brenden Matthews
>
>
Re: Review Request: Run Hadoop tutorial binaries from within build dir.
Posted by Benjamin Hindman <be...@berkeley.edu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11109/#review20659
-----------------------------------------------------------
hadoop/TUTORIAL.sh
<https://reviews.apache.org/r/11109/#comment42652>
The problem really lies here. The tutorial should aim to explain how to build a "Mesos distribution" and that should be articulated here. In other words, this is not about "building the Mesos executor package" but it's about "creating the Mesos distribution". The comment should probably also explain that the Mesos distribution is used to run the TaskTrackers via the "mapred.mesos.executor" configuration property. In fact, given that we have "mapred.mesos.executor" as a configuration property there is no reason not to be explicit here and call the final package 'hadoop-${distribution}-mesos.tar.gz'. Let's be explicit as much as we can.
- Benjamin Hindman
On May 14, 2013, 5:57 p.m., Brenden Matthews wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11109/
> -----------------------------------------------------------
>
> (Updated May 14, 2013, 5:57 p.m.)
>
>
> Review request for mesos.
>
>
> Description
> -------
>
> From e38084684887f59bce5694bb3457a94b2da026e3 Mon Sep 17 00:00:00 2001
> From: Brenden Matthews <br...@airbnb.com>
> Date: Mon, 15 Apr 2013 14:08:01 -0700
> Subject: [PATCH 02/24] Run Hadoop tutorial binaries from within build dir.
>
> Review: https://reviews.apache.org/r/11109
> ---
> hadoop/TUTORIAL.sh | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
>
> Diffs
> -----
>
> hadoop/TUTORIAL.sh d81c55562d217e8784745d4a8cedf78ada9f5453
>
> Diff: https://reviews.apache.org/r/11109/diff/
>
>
> Testing
> -------
>
> Used in production at airbnb.
>
>
> Thanks,
>
> Brenden Matthews
>
>
Re: Review Request: Run Hadoop tutorial binaries from within build dir.
Posted by Brenden Matthews <br...@diddyinc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11109/
-----------------------------------------------------------
(Updated May 14, 2013, 5:57 p.m.)
Review request for mesos.
Description
-------
>From e38084684887f59bce5694bb3457a94b2da026e3 Mon Sep 17 00:00:00 2001
From: Brenden Matthews <br...@airbnb.com>
Date: Mon, 15 Apr 2013 14:08:01 -0700
Subject: [PATCH 02/24] Run Hadoop tutorial binaries from within build dir.
Review: https://reviews.apache.org/r/11109
---
hadoop/TUTORIAL.sh | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
Diffs (updated)
-----
hadoop/TUTORIAL.sh d81c55562d217e8784745d4a8cedf78ada9f5453
Diff: https://reviews.apache.org/r/11109/diff/
Testing
-------
Used in production at airbnb.
Thanks,
Brenden Matthews