@Before/After for classes run with the test suites

[Dan LaRocque <da...@hopcount.org>]

Hi,

Bottom line up front: how can graph implementations define JUnit-style
@Before/After(Class) methods that will be respected by the TP3 test
suite runners?

To be more specific:

Let's say I'm testing a FooGraph implementation with external process
dependencies that can be idiomatically managed through JUnit's
@BeforeClass and @AfterClass method annotations.  I write a test class
like this to run TP3's process suite on FooGraph:

@RunWith(ProcessStandardSuite.class)
@ProcessStandardSuite.GraphProviderClass(provider = FooProvider.class,
graph = FooGraph.class)
public class FooProcessTest {

    // Never executed
    @BeforeClass public static void setUp() { /* stuff */ }
    @AfterClass public static void tearDown() { /* inverse of stuff */ }
}

@BeforeClass can be pretendulated by overriding methods on FooProvider,
and I'm already doing that, but it's ugly.  @AfterClass seems harder.  I
don't see how to piggyback on a method intended for some other purpose
like I'm doing with @BeforeClass.

It would also be nice to have @After and @Before or equivalents, though
I don't think it's as important as the class variants.

I see AbstractGremlinSuite.beforeTestExecution and .afterTestExecution. 
I think (not sure) that these are only intended for use within TP3.  For
instance, I see where ProcessComputerSuite overrides beforeTestExecution
to sanity-check @UseEngines, but I don't see where that functionality is
exposed to 3rd-party graph implementations running the suites, like
FooProcessTest above.

Is there already a way to do this?

thanks,
Dan

Re: @Before/After for classes run with the test suites

[Stephen Mallette <sp...@gmail.com>]

gotcha - well some good news.  as of the latest code in SNAPSHOT,
validateStructureInterfacesRegistered has been removed.  that step is no
longer necessary as we no longer try to enforce what was known as the
@Helper annotation (which was generally a bad idea anyway - if you don't
know what it was, be thankful).  So as it stands, AbstractGremlinSuite
should fire its constructor first where the GraphProvider is created, then
the @BeforeClass event in your test case and then the GraphProvider will
create Graph instances as needed, followed by @AfterClass.  Sound right?

On Wed, Apr 29, 2015 at 2:15 PM, Dan LaRocque <da...@hopcount.org> wrote:

> On Wed, Apr 29, 2015, at 09:55, Stephen Mallette wrote:
> > I tried this simple test on TinkerGraphStructureStandardTest and it
> > properly printed to the console (i.e. one line for setup and one for
> > teardown).  I'm not sure that didn't work well for you.  Am I missing
> > something?
>
> You're right, @BeforeClass *is* executed.  I was wrong when I wrote that
> it wasn't executed, though there was something subtler going on.  Here's
> what was actually happening: AbstractGremlinSuite's constructor invokes
> validateStructureInterfacesRegistered, which in turn tells the graph
> provider to open a test graph.  This happens prior to @BeforeClass
> execution.  If my external process (e.g. HBase, Cassandra, ES, whatever)
> was supposed to be started in @BeforeClass, then the test graph used in
> validation fails to open.
>
> It would be nice, though not necessary, to have @BeforeClass run prior
> to attempting to open any graphs, including the one used in validation.
> It's not necessary since I can continue putting my setup logic into the
> graph provider methods that return configuration objects, foregoing
> @BeforeClass, though that's kind of ugly.  More importantly, I think
> this removes my perceived stumbling block about @AfterClass, since I
> doubt there will be any subtle ordering issues like this in teardown.
>
> thanks,
> Dan
>

Re: @Before/After for classes run with the test suites

[Dan LaRocque <da...@hopcount.org>]

On Wed, Apr 29, 2015, at 09:55, Stephen Mallette wrote:
> I tried this simple test on TinkerGraphStructureStandardTest and it
> properly printed to the console (i.e. one line for setup and one for
> teardown).  I'm not sure that didn't work well for you.  Am I missing
> something?

You're right, @BeforeClass *is* executed.  I was wrong when I wrote that
it wasn't executed, though there was something subtler going on.  Here's
what was actually happening: AbstractGremlinSuite's constructor invokes
validateStructureInterfacesRegistered, which in turn tells the graph
provider to open a test graph.  This happens prior to @BeforeClass
execution.  If my external process (e.g. HBase, Cassandra, ES, whatever)
was supposed to be started in @BeforeClass, then the test graph used in
validation fails to open.

It would be nice, though not necessary, to have @BeforeClass run prior
to attempting to open any graphs, including the one used in validation. 
It's not necessary since I can continue putting my setup logic into the
graph provider methods that return configuration objects, foregoing
@BeforeClass, though that's kind of ugly.  More importantly, I think
this removes my perceived stumbling block about @AfterClass, since I
doubt there will be any subtle ordering issues like this in teardown.

thanks,
Dan

Re: @Before/After for classes run with the test suites

[Stephen Mallette <sp...@gmail.com>]

> in retrospect, my subject line is confusing.  I don't really care about
@After/@Before nearly as much as I care about @AfterClass/@BeforeClass.

I was following you - no problem.

> For example: my graph implementation might need to start an external HBase
process, read and write to it, and then shut it down when the test class is
done.

i figured you were trying to fire up elasticsearch or cassandra or the
like...cool

> I expect the semantics to be identical to the equivalent JUnit annotation.
In the case of @AfterClass/@BeforeClass, as I understand them, the
semantics would be your final suggestion: once for the suite itself.

I think what you're asking for is already there:

http://stackoverflow.com/q/82949/1831717

I tried this simple test on TinkerGraphStructureStandardTest and it
properly printed to the console (i.e. one line for setup and one for
teardown).  I'm not sure that didn't work well for you.  Am I missing
something?

> Also, I only need them to run if at least one test method inthe suite was
eligible for execution after application of any filters.

Unfortunately those semantics don't seem to apply to the suite execution as
the filtering for tests wont' occur @BeforeClass (i'm guessing that this is
executed statically prior to instantiation of the FooStructureStandardTest
itself.  I do think however a better job can be done around GraphProvider
not instantiating graphs for ignore tests (maybe), which will save on test
execution time. That idea was provided by Bryn here:
https://issues.apache.org/jira/browse/TINKERPOP3-641



On Tue, Apr 28, 2015 at 7:56 PM, Dan LaRocque <da...@hopcount.org> wrote:

> On Tue, Apr 28, 2015, at 19:40, Stephen Mallette wrote:
> > What do you intend to run in the @After/Before annotated methods?
> > is it something not specific to construction of the Graph instance
> > itself which is handled by the GraphProvider?  Do you expect it to
> > run per test class in the suite? or once for the suite itself?
>
> I want to start an external process in a @BeforeClass and tear it down
> in an @AfterClass.  In retrospect, my subject line is confusing.  I
> don't really care about @After/@Before nearly as much as I care about
> @AfterClass/@BeforeClass.  I tried to hash that out in the message body,
> but it was near the end.
>
> For example: my graph implementation might need to start an external
> HBase process, read and write to it, and then shut it down when the test
> class is done.
>
> The absence of @BeforeClass can be kind of worked around by hacking the
> GraphProvider, and I'm already doing that.  But it's inelegant.
> Besides, I don't know how to hack around the absence of @AfterClass.
>
> As for semantics -- "Do you expect it to run per test class in the
> suite" -- I expect the semantics to be identical to the equivalent JUnit
> annotation.  In the case of @AfterClass/@BeforeClass, as I understand
> them, the semantics would be your final suggestion: once for the suite
> itself.  Also, I only need them to run if at least one test method in
> the suite was eligible for execution after application of any filters.
>
> thanks,
> Dan
>

Re: @Before/After for classes run with the test suites

[Dan LaRocque <da...@hopcount.org>]

On Tue, Apr 28, 2015, at 19:40, Stephen Mallette wrote:
> What do you intend to run in the @After/Before annotated methods?
> is it something not specific to construction of the Graph instance 
> itself which is handled by the GraphProvider?  Do you expect it to 
> run per test class in the suite? or once for the suite itself?

I want to start an external process in a @BeforeClass and tear it down
in an @AfterClass.  In retrospect, my subject line is confusing.  I
don't really care about @After/@Before nearly as much as I care about
@AfterClass/@BeforeClass.  I tried to hash that out in the message body,
but it was near the end.

For example: my graph implementation might need to start an external
HBase process, read and write to it, and then shut it down when the test
class is done.

The absence of @BeforeClass can be kind of worked around by hacking the
GraphProvider, and I'm already doing that.  But it's inelegant. 
Besides, I don't know how to hack around the absence of @AfterClass.

As for semantics -- "Do you expect it to run per test class in the
suite" -- I expect the semantics to be identical to the equivalent JUnit
annotation.  In the case of @AfterClass/@BeforeClass, as I understand
them, the semantics would be your final suggestion: once for the suite
itself.  Also, I only need them to run if at least one test method in
the suite was eligible for execution after application of any filters.

thanks,
Dan

Re: @Before/After for classes run with the test suites

[Stephen Mallette <sp...@gmail.com>]

Hey Dan, I haven't ever tried the standard junit annotations on a test
implementation using the TinkerPop suites as runners.  I guess they don't
work at all since you're asking this question. Also regarding:

> I see AbstractGremlinSuite.beforeTestExecution and .afterTestExecution. I think
(not sure) that these are only intended for use within TP3

That's correct - those are for internal use for actual tests.  What do you
intend to run in the @After/Before annotated methods?  is it something not
specific to construction of the Graph instance itself which is handled by
the GraphProvider?  Do you expect it to run per test class in the suite? or
once for the suite itself?



On Tue, Apr 28, 2015 at 3:52 PM, Dan LaRocque <da...@hopcount.org> wrote:

> Hi,
>
> Bottom line up front: how can graph implementations define JUnit-style
> @Before/After(Class) methods that will be respected by the TP3 test
> suite runners?
>
> To be more specific:
>
> Let's say I'm testing a FooGraph implementation with external process
> dependencies that can be idiomatically managed through JUnit's
> @BeforeClass and @AfterClass method annotations.  I write a test class
> like this to run TP3's process suite on FooGraph:
>
> @RunWith(ProcessStandardSuite.class)
> @ProcessStandardSuite.GraphProviderClass(provider = FooProvider.class,
> graph = FooGraph.class)
> public class FooProcessTest {
>
>     // Never executed
>     @BeforeClass public static void setUp() { /* stuff */ }
>     @AfterClass public static void tearDown() { /* inverse of stuff */ }
> }
>
> @BeforeClass can be pretendulated by overriding methods on FooProvider,
> and I'm already doing that, but it's ugly.  @AfterClass seems harder.  I
> don't see how to piggyback on a method intended for some other purpose
> like I'm doing with @BeforeClass.
>
> It would also be nice to have @After and @Before or equivalents, though
> I don't think it's as important as the class variants.
>
> I see AbstractGremlinSuite.beforeTestExecution and .afterTestExecution.
> I think (not sure) that these are only intended for use within TP3.  For
> instance, I see where ProcessComputerSuite overrides beforeTestExecution
> to sanity-check @UseEngines, but I don't see where that functionality is
> exposed to 3rd-party graph implementations running the suites, like
> FooProcessTest above.
>
> Is there already a way to do this?
>
> thanks,
> Dan
>