Issues when running multiple suites with Maven's reuseForks=true

[Dan LaRocque <da...@hopcount.org>]

Hi,

When I execute a set of graph implementation test classes that @RunWith
various descendants of AbstractGremlinSuite in Maven, some tests that
are designed for and limited to an OLAP compute engine actually run with
an OLTP compute engine.  This behavior disappears when one suite
containing an affected test runs alone.

Setting maven-surefire-plugin's reuseForks=false makes these bizarre
failures disappear.  There's no parallel execution in either of the
configs I've tried.  The only difference is whether each class gets its
own pristine JVM or whether the classes share a single JVM.  That makes
me suspect some kind of interference between suites, perhaps through
static state.

Consider AbstractGremlinSuite's constructor.  It invokes some static
methods on GraphManager that store the compute engine type and graph
provider in static fields on GraphManager.  If Maven happened to
construct multiple test suite instances prior to running their test
methods, these types of bizarre compute engine mismatches seem like they
could follow.  I'm merely speculating on that last point though; I don't
really have any idea how it orders suite construction and test method
execution. 

Is this working as intended (i.e. "stop reusing forks, problem solved")?
 Or is this something we should investigate?

thanks,
Dan

Re: Issues when running multiple suites with Maven's reuseForks=true

[Stephen Mallette <sp...@gmail.com>]

I'm not quite sure what's going on with that.  no idea why we would need a
fresh JVM for each test class, but i would guess that it has something to
do with your line of thinking: confusion in the GraphManager class.  Of
course, some simple tests seem to show that the suites are constructed in a
serial fashion right before they are executed, so I'm not sure where
GraphManager would get confused.  I'd rather not force reuseForks=false if
we don't need to, but I can't think of what to try to fix it.....

On Thu, Apr 30, 2015 at 1:12 AM, Dan LaRocque <da...@hopcount.org> wrote:

> Hi,
>
> When I execute a set of graph implementation test classes that @RunWith
> various descendants of AbstractGremlinSuite in Maven, some tests that
> are designed for and limited to an OLAP compute engine actually run with
> an OLTP compute engine.  This behavior disappears when one suite
> containing an affected test runs alone.
>
> Setting maven-surefire-plugin's reuseForks=false makes these bizarre
> failures disappear.  There's no parallel execution in either of the
> configs I've tried.  The only difference is whether each class gets its
> own pristine JVM or whether the classes share a single JVM.  That makes
> me suspect some kind of interference between suites, perhaps through
> static state.
>
> Consider AbstractGremlinSuite's constructor.  It invokes some static
> methods on GraphManager that store the compute engine type and graph
> provider in static fields on GraphManager.  If Maven happened to
> construct multiple test suite instances prior to running their test
> methods, these types of bizarre compute engine mismatches seem like they
> could follow.  I'm merely speculating on that last point though; I don't
> really have any idea how it orders suite construction and test method
> execution.
>
> Is this working as intended (i.e. "stop reusing forks, problem solved")?
>  Or is this something we should investigate?
>
> thanks,
> Dan
>