You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Brenden Matthews <br...@diddyinc.com> on 2013/04/17 00:08:46 UTC
Review Request: Use more portable paths for tutorial test.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10563/
-----------------------------------------------------------
Review request for mesos.
Description
-------
Use more portable paths for tutorial test.
Diffs
-----
hadoop/TUTORIAL.sh f8131cd
Diff: https://reviews.apache.org/r/10563/diff/
Testing
-------
Used in production at Airbnb.
Thanks,
Brenden Matthews
Re: Review Request: Use more portable paths for tutorial test.
Posted by Ben Mahler <be...@gmail.com>.
> On April 17, 2013, 5:53 p.m., Ben Mahler wrote:
> > hadoop/TUTORIAL.sh, line 611
> > <https://reviews.apache.org/r/10563/diff/1/?file=281696#file281696line611>
> >
> > I would argue this is less "portable" as you're using a static path in /tmp.
> >
> > Can you clarify as to why this has anything to do with portability?
>
> Brenden Matthews wrote:
> You're making the assumption that writing to 'out' is okay. This fails in my testing. /tmp is generally a safe place for temporary files on unix-y systems.
Why does this fail on your system? We generally avoid using static paths, even in /tmp, because if multiple users are running the tutorial on the machine, they will stomp on each others files.
Can we get away with s/out/TUTORIAL.out/ ?
- Ben
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10563/#review19336
-----------------------------------------------------------
On April 16, 2013, 10:08 p.m., Brenden Matthews wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10563/
> -----------------------------------------------------------
>
> (Updated April 16, 2013, 10:08 p.m.)
>
>
> Review request for mesos.
>
>
> Description
> -------
>
> Use more portable paths for tutorial test.
>
>
> Diffs
> -----
>
> hadoop/TUTORIAL.sh f8131cd
>
> Diff: https://reviews.apache.org/r/10563/diff/
>
>
> Testing
> -------
>
> Used in production at Airbnb.
>
>
> Thanks,
>
> Brenden Matthews
>
>
Re: Review Request: Use more portable paths for tutorial test.
Posted by Brenden Matthews <br...@diddyinc.com>.
> On April 17, 2013, 5:53 p.m., Ben Mahler wrote:
> > hadoop/TUTORIAL.sh, line 611
> > <https://reviews.apache.org/r/10563/diff/1/?file=281696#file281696line611>
> >
> > I would argue this is less "portable" as you're using a static path in /tmp.
> >
> > Can you clarify as to why this has anything to do with portability?
You're making the assumption that writing to 'out' is okay. This fails in my testing. /tmp is generally a safe place for temporary files on unix-y systems.
- Brenden
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10563/#review19336
-----------------------------------------------------------
On April 16, 2013, 10:08 p.m., Brenden Matthews wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10563/
> -----------------------------------------------------------
>
> (Updated April 16, 2013, 10:08 p.m.)
>
>
> Review request for mesos.
>
>
> Description
> -------
>
> Use more portable paths for tutorial test.
>
>
> Diffs
> -----
>
> hadoop/TUTORIAL.sh f8131cd
>
> Diff: https://reviews.apache.org/r/10563/diff/
>
>
> Testing
> -------
>
> Used in production at Airbnb.
>
>
> Thanks,
>
> Brenden Matthews
>
>
Re: Review Request: Use more portable paths for tutorial test.
Posted by Ben Mahler <be...@gmail.com>.
> On April 17, 2013, 5:53 p.m., Ben Mahler wrote:
> > hadoop/TUTORIAL.sh, line 611
> > <https://reviews.apache.org/r/10563/diff/1/?file=281696#file281696line611>
> >
> > I would argue this is less "portable" as you're using a static path in /tmp.
> >
> > Can you clarify as to why this has anything to do with portability?
>
> Brenden Matthews wrote:
> You're making the assumption that writing to 'out' is okay. This fails in my testing. /tmp is generally a safe place for temporary files on unix-y systems.
>
> Ben Mahler wrote:
> Why does this fail on your system? We generally avoid using static paths, even in /tmp, because if multiple users are running the tutorial on the machine, they will stomp on each others files.
>
> Can we get away with s/out/TUTORIAL.out/ ?
>
> Brenden Matthews wrote:
> The build directory is not owned by the mapred user, so it fails. I usually build as root and run Hadoop as mapred or hdfs.
I see, this sounds like an issue stemming from the fact that you're using different users to build / run, rather than portability issues like the summary originally suggested?
- Ben
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10563/#review19336
-----------------------------------------------------------
On April 16, 2013, 10:08 p.m., Brenden Matthews wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10563/
> -----------------------------------------------------------
>
> (Updated April 16, 2013, 10:08 p.m.)
>
>
> Review request for mesos.
>
>
> Description
> -------
>
> Use more portable paths for tutorial test.
>
>
> Diffs
> -----
>
> hadoop/TUTORIAL.sh f8131cd
>
> Diff: https://reviews.apache.org/r/10563/diff/
>
>
> Testing
> -------
>
> Used in production at Airbnb.
>
>
> Thanks,
>
> Brenden Matthews
>
>
Re: Review Request: Use more portable paths for tutorial test.
Posted by Brenden Matthews <br...@diddyinc.com>.
> On April 17, 2013, 5:53 p.m., Ben Mahler wrote:
> > hadoop/TUTORIAL.sh, line 611
> > <https://reviews.apache.org/r/10563/diff/1/?file=281696#file281696line611>
> >
> > I would argue this is less "portable" as you're using a static path in /tmp.
> >
> > Can you clarify as to why this has anything to do with portability?
>
> Brenden Matthews wrote:
> You're making the assumption that writing to 'out' is okay. This fails in my testing. /tmp is generally a safe place for temporary files on unix-y systems.
>
> Ben Mahler wrote:
> Why does this fail on your system? We generally avoid using static paths, even in /tmp, because if multiple users are running the tutorial on the machine, they will stomp on each others files.
>
> Can we get away with s/out/TUTORIAL.out/ ?
>
> Brenden Matthews wrote:
> The build directory is not owned by the mapred user, so it fails. I usually build as root and run Hadoop as mapred or hdfs.
>
> Ben Mahler wrote:
> I see, this sounds like an issue stemming from the fact that you're using different users to build / run, rather than portability issues like the summary originally suggested?
That's one way to put it.
If I try and run the jobtracker as `root' it fails because Hadoop doesn't like running as root.
- Brenden
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10563/#review19336
-----------------------------------------------------------
On April 16, 2013, 10:08 p.m., Brenden Matthews wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10563/
> -----------------------------------------------------------
>
> (Updated April 16, 2013, 10:08 p.m.)
>
>
> Review request for mesos.
>
>
> Description
> -------
>
> Use more portable paths for tutorial test.
>
>
> Diffs
> -----
>
> hadoop/TUTORIAL.sh f8131cd
>
> Diff: https://reviews.apache.org/r/10563/diff/
>
>
> Testing
> -------
>
> Used in production at Airbnb.
>
>
> Thanks,
>
> Brenden Matthews
>
>
Re: Review Request: Use more portable paths for tutorial test.
Posted by Brenden Matthews <br...@diddyinc.com>.
> On April 17, 2013, 5:53 p.m., Ben Mahler wrote:
> > hadoop/TUTORIAL.sh, line 611
> > <https://reviews.apache.org/r/10563/diff/1/?file=281696#file281696line611>
> >
> > I would argue this is less "portable" as you're using a static path in /tmp.
> >
> > Can you clarify as to why this has anything to do with portability?
>
> Brenden Matthews wrote:
> You're making the assumption that writing to 'out' is okay. This fails in my testing. /tmp is generally a safe place for temporary files on unix-y systems.
>
> Ben Mahler wrote:
> Why does this fail on your system? We generally avoid using static paths, even in /tmp, because if multiple users are running the tutorial on the machine, they will stomp on each others files.
>
> Can we get away with s/out/TUTORIAL.out/ ?
The build directory is not owned by the mapred user, so it fails. I usually build as root and run Hadoop as mapred or hdfs.
- Brenden
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10563/#review19336
-----------------------------------------------------------
On April 16, 2013, 10:08 p.m., Brenden Matthews wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10563/
> -----------------------------------------------------------
>
> (Updated April 16, 2013, 10:08 p.m.)
>
>
> Review request for mesos.
>
>
> Description
> -------
>
> Use more portable paths for tutorial test.
>
>
> Diffs
> -----
>
> hadoop/TUTORIAL.sh f8131cd
>
> Diff: https://reviews.apache.org/r/10563/diff/
>
>
> Testing
> -------
>
> Used in production at Airbnb.
>
>
> Thanks,
>
> Brenden Matthews
>
>
Re: Review Request: Use more portable paths for tutorial test.
Posted by Ben Mahler <be...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10563/#review19336
-----------------------------------------------------------
hadoop/TUTORIAL.sh
<https://reviews.apache.org/r/10563/#comment40015>
I would argue this is less "portable" as you're using a static path in /tmp.
Can you clarify as to why this has anything to do with portability?
- Ben Mahler
On April 16, 2013, 10:08 p.m., Brenden Matthews wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10563/
> -----------------------------------------------------------
>
> (Updated April 16, 2013, 10:08 p.m.)
>
>
> Review request for mesos.
>
>
> Description
> -------
>
> Use more portable paths for tutorial test.
>
>
> Diffs
> -----
>
> hadoop/TUTORIAL.sh f8131cd
>
> Diff: https://reviews.apache.org/r/10563/diff/
>
>
> Testing
> -------
>
> Used in production at Airbnb.
>
>
> Thanks,
>
> Brenden Matthews
>
>