Re: scoping integration tests

[Roman Shaposhnik <rv...@apache.org>]

+bigtop-dev

On Wed, Nov 30, 2011 at 10:11 AM, Stack <st...@duboce.net> wrote:
> Chatting w/ Roman and Mikhail yesterday at the Hackathon, and with a
> probably broader goals than perhaps what you've been thinking on
> Jesse, I think:
>
> + IntegrationTests* -- where IntegrationTest* is tests with the
> IntegrationTest prefix and are that run when  'mvn verify' as per your
> recent contrib Jesse -- differ from unit tests in that they always run
> against a cluster

Small correction: I don't "always run against a cluster" is a useful
assumption here. In fact, I really want those tests to be completly
agnostic of whether they run against a Mini, Pseudo-distributed
or a fully distributed cluster.

> + IntegrationTest* do not concern themselves with cluster setup and
> teardown -- this is done external to the test code by the failsafe
> plugin + configs (If we do this and the previous point, then our
> IntegrationTest* can be run by bigtop if I understood Roman properly
> yesterday).

Correct. An excellent example of how this can be done is pigsmoke
artifact with its -Dpigunit.exectype.cluster=true knob (and a few
others):
     https://issues.apache.org/jira/browse/PIG-1799

> + Cluster setup and teardown needs to be pluggable (it seems like
> failsafe can do this for us).

Only if somebody puts an effort of actually implementing a
deployment plugin (perhaps based on Whirr?). This is a non-trivial
exercise (although it is a nice project).

> I don't want to get hung up on nomenclature.  If there are long email
> threads on test taxonomy, then something has gone awry IMO.  A clean
> surefire for unit tests and failsafe for integration tests where unit
> tests are whats runnable w/o need of outside dependencies -- e.g. an
> idling hbase or mr cluster -- and is what devs and our CI does seems
> like a pretty clean separation to me, one that even us hbase devs
> might grok.
>
> What do you reckon?

+1

An extra point that I was trying to articulate very strongly to the GB guys
yesterday is this: we need tests NOW even when we don't have the
rest of the infrastructure in place (mvn verify, deployment, separation
of tests, etc.) If they can start giving me access to the tests they've
created I can start running them in Bigtop today.

Sure they will be "one step removed" from a casual HBase developer
but the bugs that I'm going to file will be quite real.

Thanks,
Roman.