[jira] [Resolved] (HBASE-23780) Edit of test classifications

["Michael Stack (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-23780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael Stack resolved HBASE-23780.
-----------------------------------
    Fix Version/s: master
                   2.3.0
     Hadoop Flags: Reviewed
         Assignee: Michael Stack
       Resolution: Fixed

Pushed this on branch-2 and master. Reclassifies tests according to how long they take in recent runs.

Idea is to move as many tests as possible into small category where they all run in the one JVM. Allows us to move through a bunch in a short time.

Next up is getting tests into the medium category. These are tests that take < 50 seconds. Parallelization should help here as they don't stick around too long. TODO is fix tests that only run a single test and the test takes little time... Aggregate up single tests into one test suite. Medium tests start a JVM. This costs. Amortize.

Large tests we'll just have to tackle one at a time.

I've been using a coworker's script to up the parallelization:

{code}$ x=1.0 mvn -T $xC -Dsurefire.firstPartForkCount=$xC -Dsurefire.secondPartForkCount=$xC -PrunSmallTests{code} ... etc.

The -T in particular helps by making maven use more than one thread. 



> Edit of test classifications
> ----------------------------
>
>                 Key: HBASE-23780
>                 URL: https://issues.apache.org/jira/browse/HBASE-23780
>             Project: HBase
>          Issue Type: Bug
>          Components: test
>            Reporter: Michael Stack
>            Assignee: Michael Stack
>            Priority: Major
>             Fix For: 2.3.0, master
>
>
> Our test classifications have drifted. You can see for yourself running each of the small/medium and large test suites. See test complete times. See how even some large tests should be small and vice versa.
> The more small tests we can run inside the single JVM, the faster we'll get through the build. Tests that are Medium start their own JVM for each test. Tests that are Medium but only last a second or two are expensive and should be aggregated with other single, short tests to amortize the JVM startup.
> Anyways, let me edit the test categories and try and clean them up some.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)