Test workflow - blacklist entire suites and run any independently

[Adam Roberts <AR...@uk.ibm.com>]

Hi, is there an existing way to blacklist any test suite?

Ideally we'd have a text file with a series of names (let's say comma 
separated) and if a name matches with the fully qualified class name for a 
suite, this suite will be skipped.

Perhaps we can achieve this via ScalaTest or Maven?

Currently if a number of suites are failing we're required to comment 
these out, commit and push this change then kick off a Jenkins job 
(perhaps building a custom branch) - not ideal when working with Jenkins, 
would be quicker to use such a mechanism as described above as opposed to 
having a few branches that are a little different from others.

Also, how can we quickly only run any one suite within, say, sql/hive? -f 
sql/hive/pom.xml with -nsu results in compile failures each time.
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Re: Test workflow - blacklist entire suites and run any independently

[Adam Roberts <AR...@uk.ibm.com>]

Thanks Josh, I should have added that we've tried with -DwildcardSuites 
and Maven and we use this helpful feature regularly (although this does 
result in building plenty of tests and running other tests in other 
modules too), so wondering if there's a more "streamlined" way - e.g. with 
junit and eclipse we'd just right click one individual unit test and 
that'd be run - without building again AFAIK

Unfortunately using sbt causes a lot of pain, such as...

[error]
[error]   last tree to typer: 
Literal(Constant(org.apache.spark.sql.test.ExamplePoint))
[error]               symbol: null
[error]    symbol definition: null
[error]                  tpe: 
Class(classOf[org.apache.spark.sql.test.ExamplePoint])
[error]        symbol owners:
[error]       context owners: class ExamplePointUDT -> package test
[error]

and then an awfully long stacktrace with plenty of errors. Must be an 
easier way...



From:   Josh Rosen <ro...@gmail.com>
To:     Adam Roberts/UK/IBM@IBMGB
Cc:     dev <de...@spark.apache.org>
Date:   21/09/2015 19:19
Subject:        Re: Test workflow - blacklist entire suites and run any 
independently



For quickly running individual suites: 
https://cwiki.apache.org/confluence/display/SPARK/Useful+Developer+Tools#UsefulDeveloperTools-RunningIndividualTests

On Mon, Sep 21, 2015 at 8:21 AM, Adam Roberts <AR...@uk.ibm.com> wrote:
Hi, is there an existing way to blacklist any test suite? 

Ideally we'd have a text file with a series of names (let's say comma 
separated) and if a name matches with the fully qualified class name for a 
suite, this suite will be skipped. 

Perhaps we can achieve this via ScalaTest or Maven? 

Currently if a number of suites are failing we're required to comment 
these out, commit and push this change then kick off a Jenkins job 
(perhaps building a custom branch) - not ideal when working with Jenkins, 
would be quicker to use such a mechanism as described above as opposed to 
having a few branches that are a little different from others. 

Also, how can we quickly only run any one suite within, say, sql/hive? -f 
sql/hive/pom.xml with -nsu results in compile failures each time.
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Re: Test workflow - blacklist entire suites and run any independently

[Josh Rosen <ro...@gmail.com>]

For quickly running individual suites:
https://cwiki.apache.org/confluence/display/SPARK/Useful+Developer+Tools#UsefulDeveloperTools-RunningIndividualTests

On Mon, Sep 21, 2015 at 8:21 AM, Adam Roberts <AR...@uk.ibm.com> wrote:

> Hi, is there an existing way to blacklist any test suite?
>
> Ideally we'd have a text file with a series of names (let's say comma
> separated) and if a name matches with the fully qualified class name for a
> suite, this suite will be skipped.
>
> Perhaps we can achieve this via ScalaTest or Maven?
>
> Currently if a number of suites are failing we're required to comment
> these out, commit and push this change then kick off a Jenkins job (perhaps
> building a custom branch) - not ideal when working with Jenkins, would be
> quicker to use such a mechanism as described above as opposed to having a
> few branches that are a little different from others.
>
> Also, how can we quickly only run any one suite within, say, sql/hive? -f
> sql/hive/pom.xml with -nsu results in compile failures each time.
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>