Minimum RAM for running full pig unit test suite?

[Andrew Bayer <an...@gmail.com>]

Hey all -

I'm trying to figure out the right machine to run regular full suites of
Pig unit tests on. I've been doing pretty well with an 8gb RAM VM, but have
now hit a case
of org.apache.pig.test.TestSkewedJoin.testSkewedJoinManyReducers failing,
claiming that it timed out (taking all of 0.0020 seconds to do so) and not
running any other tests from TestSkewedJoin. Ant's error message is
obviously not very useful, but some digging suggests that Ant may be
throwing this when the forked JVM for the tests fails to launch or crashes
due to memory issues.

Is anyone actually running the full suite of unit tests with reliable,
consistent results, and if so, how much RAM does the machine you're running
them on have? And has anyone seen anything like this failure before or have
any other suggestions for what the root cause may be? I can't reproduce it
with any consistency.

A.

Re: Minimum RAM for running full pig unit test suite?

[Andrew Bayer <an...@gmail.com>]

Ok, turns out the test *did* run for more than 0.0020 seconds. That's just
junit lying. It seems to end up running 'til the Junit timeout hits. It's
worth noting that the same test takes around 3-4 minutes on a box with 48gb
RAM.

A.

On Wed, Feb 22, 2012 at 1:32 PM, Andrew Bayer <an...@gmail.com>wrote:

> Hey all -
>
> I'm trying to figure out the right machine to run regular full suites of
> Pig unit tests on. I've been doing pretty well with an 8gb RAM VM, but have
> now hit a case
> of org.apache.pig.test.TestSkewedJoin.testSkewedJoinManyReducers failing,
> claiming that it timed out (taking all of 0.0020 seconds to do so) and not
> running any other tests from TestSkewedJoin. Ant's error message is
> obviously not very useful, but some digging suggests that Ant may be
> throwing this when the forked JVM for the tests fails to launch or crashes
> due to memory issues.
>
> Is anyone actually running the full suite of unit tests with reliable,
> consistent results, and if so, how much RAM does the machine you're running
> them on have? And has anyone seen anything like this failure before or have
> any other suggestions for what the root cause may be? I can't reproduce it
> with any consistency.
>
> A.
>