You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Adam Monsen <ha...@gmail.com> on 2013/02/03 16:06:23 UTC
Re: introduction: Adam Monsen
Ben, Vinod, thanks again for your help!
I'm busy for the next couple days, but I'll take a look at this again
soon. More replies inline, below.
On 01/30/2013 02:39 PM, Benjamin Mahler wrote:
> Hi Adam, the build issue on trunk that your seeing is because we don't
> yet support building with GCC 4.7.
>
> I've been keeping track of the particular boost issue here:
> https://issues.apache.org/jira/browse/MESOS-271
> https://svn.boost.org/trac/boost/ticket/6854
>
> If your looking to use git, the svn repo is mirrored here:
> git://git.apache.org/mesos.git <http://git.apache.org/mesos.git>
>
> Note that you should track "trunk" instead of "master".
Great info, thank you!
> P.S. If you're interested in Hadoop on mesos, you may also be interested
> in Storm or Spark on Mesos, which are maintained externally:
>
> https://github.com/nathanmarz/storm-mesos
> https://github.com/mesos/spark
Wow, both of these projects look very compelling! I just started
watching Nathan Marz's talk on Storm. I think I might try that out
before Hadoop.
Re: MESOS-347 cgroups test failure
Posted by Adam Monsen <ha...@gmail.com>.
Yes, that repro breaks for me exactly as you describe.
On 02/12/2013 05:12 PM, Benjamin Hindman wrote:
> It looks like we've reproduced locally. You can confirm by doing your
> cleanup, then running (repeatedly) with
> --gtest_filter='CgroupsNoHierarchyTest.*'. Now try running with
> --gtest_filter='Cgroups*BalloonFramework'. Try
> with --gtest_filter='CgroupsNoHierarchyTest.*' again, and if it fails we
> have a winner!
Re: MESOS-347 cgroups test failure (was: introduction: Adam Monsen)
Posted by Benjamin Hindman <be...@eecs.berkeley.edu>.
It looks like we've reproduced locally. You can confirm by doing your
cleanup, then running (repeatedly) with
--gtest_filter='CgroupsNoHierarchyTest.*'. Now try running with
--gtest_filter='Cgroups*BalloonFramework'. Try
with --gtest_filter='CgroupsNoHierarchyTest.*' again, and if it fails we
have a winner!
On Tue, Feb 12, 2013 at 4:41 PM, Adam Monsen <ha...@gmail.com> wrote:
> On 02/11/2013 02:11 PM, Benjamin Hindman wrote:
> > The error implies you still have a mounted hierarchy somewhere. Or we
> have
> > a bug and haven't properly cleaned up our own hierarchies causing this
> test
> > to fail.
>
> I think I did proper cleanup, so I'm guessing the latter.
>
> https://issues.apache.org/jira/browse/MESOS-347
>
> Can anyone else (on Linux) repro this bug?
>
>
MESOS-347 cgroups test failure (was: introduction: Adam Monsen)
Posted by Adam Monsen <ha...@gmail.com>.
On 02/11/2013 02:11 PM, Benjamin Hindman wrote:
> The error implies you still have a mounted hierarchy somewhere. Or we have
> a bug and haven't properly cleaned up our own hierarchies causing this test
> to fail.
I think I did proper cleanup, so I'm guessing the latter.
https://issues.apache.org/jira/browse/MESOS-347
Can anyone else (on Linux) repro this bug?
Re: introduction: Adam Monsen
Posted by Benjamin Hindman <be...@eecs.berkeley.edu>.
> [----------] 1 test from CgroupsNoHierarchyTest
> [ RUN ] CgroupsNoHierarchyTest.ROOT_CGROUPS_MountUnmountHierarchy
> tests/cgroups_tests.cpp:294: Failure
> cgroups::mount(HIERARCHY, "cpu,memory"): 'cpu' is already attached to
> another hierarchy
> [ FAILED ] CgroupsNoHierarchyTest.ROOT_CGROUPS_MountUnmountHierarchy (0
> ms)
> [----------] 1 test from CgroupsNoHierarchyTest (0 ms total)
>
The error implies you still have a mounted hierarchy somewhere. Or we have
a bug and haven't properly cleaned up our own hierarchies causing this test
to fail.
Re: introduction: Adam Monsen
Posted by Adam Monsen <ha...@gmail.com>.
Ben H. wrote:
>Adam wrote:
>> I've got it down to 10 (consistently) failing tests, but they don't
>> seem too critical for my current experiments. Looks like all the test
>> failures (except for MESOS-335, which doesn't repro reliably) are
>> related to Linux cgroups.
>
> Is this on trunk?
Yep. I'm using a git clone of git://github.com/apache/mesos.git,
checked out to the "trunk" branch. git commit b8d38f0476833.
git-svn-id shows trunk r1444294.
> What are the error messages of the failed tests?
Update: I *had* 10 failed tests when running `sudo make check`, but
after cleaning up what appears to be some left over cgroups detritus
(using `umount /cgroup` and `sudo cgclear`), I now consistently get
just one failed test, CgroupsNoHierarchyTest (again, using `sudo make
check`):
- 8< -
[----------] 1 test from CgroupsNoHierarchyTest
[ RUN ] CgroupsNoHierarchyTest.ROOT_CGROUPS_MountUnmountHierarchy
tests/cgroups_tests.cpp:294: Failure
cgroups::mount(HIERARCHY, "cpu,memory"): 'cpu' is already attached to
another hierarchy
[ FAILED ] CgroupsNoHierarchyTest.ROOT_CGROUPS_MountUnmountHierarchy (0 ms)
[----------] 1 test from CgroupsNoHierarchyTest (0 ms total)
- >8 -
Interestingly, this test (CgroupsNoHierarchyTest) passes when I run it
stand-alone (using the bin/mesos-tests syntax you provided below).
> ./bin/mesos-tests --gtest_filter=CgroupsNoHierarchyTest.*
>
> Likewise, you can run all tests but the CgroupsNoHierarchyTests with:
>
> ./bin/mesos-tests --gtest_filter=-CgroupsNoHierarchyTest.*
Perfect, thank you.
Re: introduction: Adam Monsen
Posted by Benjamin Hindman <be...@eecs.berkeley.edu>.
> I've got it down to 10 (consistently) failing tests, but they don't
> seem too critical for my current experiments. Looks like all the test
> failures (except for MESOS-335, which doesn't repro reliably) are
> related to Linux cgroups.
>
Is this on trunk? What are the error messages of the failed tests?
> Semi-related question: how do I run a single unit test? Say, only
> CgroupsNoHierarchyTest?
>
./bin/mesos-tests --gtest_filter=CgroupsNoHierarchyTest.*
Likewise, you can run all tests but the CgroupsNoHierarchyTests with:
./bin/mesos-tests --gtest_filter=-CgroupsNoHierarchyTest.*
Re: introduction: Adam Monsen
Posted by Adam Monsen <ha...@gmail.com>.
I've got it down to 10 (consistently) failing tests, but they don't
seem too critical for my current experiments. Looks like all the test
failures (except for MESOS-335, which doesn't repro reliably) are
related to Linux cgroups.
https://github.com/mesos/mesos/wiki/Using-Linux-Containers was
helpful! It was bit out of date, so I updated it. With regard to the
cgroups unit tests, it said "These are disabled by default", but `make
check` runs cgroups tests for me. It also mentioned a script/program
called "alltests" that no longer exists.
Semi-related question: how do I run a single unit test? Say, only
CgroupsNoHierarchyTest?
I wrote:
> I've done little C/C++ hacking on Linux.
Really I should have said I've done little C/C++ hacking, period. But
I get the basics. :)
Re: introduction: Adam Monsen
Posted by Adam Monsen <ha...@gmail.com>.
On Fri, Feb 8, 2013 at 2:08 PM, Benjamin Mahler <bm...@twitter.com> wrote:
> For a dev environment, I'd recommend:
>
> Ubuntu 10.04 (64 bit), or
> Ubuntu 12.04 (64 bit)
>
> with our current release, we don't support gcc 4.7
> We should be compiling on gcc-4.2.
I went with 64-but Ubuntu 12.04 desktop.
I didn't find a package for gcc-4.2. Looks like the oldest available
in the default Ubuntu 12.04 repositories is gcc-4.4. I recall it being
a pain to manually build an old gcc (or maybe the pain is building the
whole GNU autotools toolchain... anyway) so I just tried the one it
came with (gcc-4.6). `make` worked fine, but `make check` didn't.
The first time I ran `make check` I ran into MESOS-335, but I've had
different results from subsequent tries... sometimes 19 tests fail,
sometimes 18.
I'm usually seeing something exactly like what is documented in
http://mail-archives.apache.org/mod_mbox/incubator-mesos-dev/201211.mbox/%3C1110106306.2231.1353966687280.JavaMail.hudson@aegis%3E
, for example: https://gist.github.com/meonkeys/4748152
I can give gcc-4.2 a shot too.
> Feel free to communicate your experiments, including any things you found
> difficult, lacking in documentation, or confusing. We love getting this kind
> of visibility into how people are using mesos.
Will do. Fair warning, I've done little C/C++ hacking on Linux.
Re: introduction: Adam Monsen
Posted by Benjamin Mahler <bm...@twitter.com>.
For a dev environment, I'd recommend:
Ubuntu 10.04 (64 bit), or
Ubuntu 12.04 (64 bit)
Unfortunately, with our current release, we don't support gcc 4.7. I
realize this is annoying, and this is something that will be fixed as part
of one of the next releases. So, using 12.04 will mean installing an older
version of gcc (for now). We should be compiling on gcc-4.2.
Feel free to communicate your experiments, including any things you found
difficult, lacking in documentation, or confusing. We love getting this
kind of visibility into how people are using mesos.
P.S. So you're also a careful curator of your email! Once we're in a direct
conversation, feel free to leave me in to/cc headers, that helps me keep
track of it :)
On Fri, Feb 8, 2013 at 1:34 PM, Adam Monsen <ha...@gmail.com> wrote:
> What would you guys recommend for a dev environment? I'll set up a
> 64-bit OS for my Mesos experiments. I'm most familiar with Ubuntu
> Desktop lately, so I'd prefer to stick with that. I'm thinking Ubuntu
> 12.10 64-bit desktop, and something other than gcc-4.7. Do I need a
> particular version of gcc/g++?
>
> I finished a simple Hadoop hello world, so look out, uh, world! :)
>
> Oh and please let me know if you guys would rather *not* be explicitly
> listed in To/Cc headers. I've been doing that ever since corresponding a
> bit on the git mailing list. I prefer it (since I rarely read all emails
> to mailing lists unless I'm in a To/Cc), but I understand if you don't.
>
>
Re: introduction: Adam Monsen
Posted by Adam Monsen <ha...@gmail.com>.
What would you guys recommend for a dev environment? I'll set up a
64-bit OS for my Mesos experiments. I'm most familiar with Ubuntu
Desktop lately, so I'd prefer to stick with that. I'm thinking Ubuntu
12.10 64-bit desktop, and something other than gcc-4.7. Do I need a
particular version of gcc/g++?
I finished a simple Hadoop hello world, so look out, uh, world! :)
Oh and please let me know if you guys would rather *not* be explicitly
listed in To/Cc headers. I've been doing that ever since corresponding a
bit on the git mailing list. I prefer it (since I rarely read all emails
to mailing lists unless I'm in a To/Cc), but I understand if you don't.