[jira] [Commented] (KAFKA-4594) Annotate integration tests and provide gradle build targets to run subsets of tests

["Ismael Juma (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-4594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15800944#comment-15800944 ] 

Ismael Juma commented on KAFKA-4594:
------------------------------------

If we go in the annotations direction, we should probably get rid of the `integration` and `unit` folders in the `core` project. Alternatively, we could keep the folder based approach and rely on that. I think I prefer the tag-based approach, particularly if we can tag superclasses/supertraits as you suggested.

> Annotate integration tests and provide gradle build targets to run subsets of tests
> -----------------------------------------------------------------------------------
>
>                 Key: KAFKA-4594
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4594
>             Project: Kafka
>          Issue Type: Bug
>          Components: unit tests
>            Reporter: Ewen Cheslack-Postava
>            Priority: Minor
>              Labels: newbie
>
> It's possible to annotate unit tests and control which ones run based on these annotations. Our tests have grown to take quite a long time because we've added a lot of relatively expensive integration tests. Because these tests are all interleaved with the true unit tests and block things like checkstyle from running, we've ended up with pretty bad turn around time on feedback on simple PR issues.
> We could make this a lot nicer, and encourage users to actually run some tests before submitting PRs with problems, by making it easier to get lightweight feedback based only on unit tests. We'd want to annotate integration tests (which can probably mostly be accomplished by annotating just a few shared {{*TestHarness}} classes and then setting up the build tasks and dependencies to run things in fast-fail, more user-friendly order. In particular, it'd be nice to:
> a) Have gradle tasks that can run just the unit tests + checkstyle. This provides devs a *much* faster way to get at least basic feedback about whether there are problems with their PR. PR Jenkins builds will also give much faster feedback (and avoid clogging up the Jenkins slaves with lots of unnecessary work that will ultimately just fail on a unit test anyway).
> b) Have the order of tests be 1) build everything and run their unit tests, 2) run checkstyle against all components, then 3) run integration tests. This could help address one of the frustrating things about checkstyle tests today, which are put after all tests to avoid annoying failures during development, but also results in a lot of folks missing simple checkstyle issues before submitting PRs.
> The two obvious options I see for this are:
> 1. In JUnit4 (which we currently use), use Categories. These seem to unfortunately still be under an experimental package. We pin to a specific version of JUnit, so this may not be a huge issue. But it does mean any future migration would be more painful as the annotations & integration with the test runner would need to change in the future.
> 2. Wait for JUnit5 with its built-in Tag support. This would be tied to making JDK8 the minimum supported Java version. Given current plans that would mean we couldn't do this until at least 0.11.0.0.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)