[jira] [Comment Edited] (HIVE-14540) Create batches for non qfile tests

["Siddharth Seth (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-14540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423191#comment-15423191 ] 

Siddharth Seth edited comment on HIVE-14540 at 8/16/16 6:23 PM:
----------------------------------------------------------------

bq. AFAIK maven is currently configured to fork a jvm for every test(resuseforks=false)..
That was a little unexpected, or not... this needs to be fixed as well.

In a ptest run reuseForks=false does not make a lot of difference (yet). Each individual test class is run via an independent mvn test invocation. That's over 600 mvn test invocations on 24 drones. The intent of batching was to amortize some of the time wasted in doing this. If we go down the batching route, reuseForks will come into play.

This sheet contains information on test runtimes - reported by junit vs reported by ptest.
https://docs.google.com/spreadsheets/d/1QYLWp61kLWXEm6MGxwFrG5caTiTREm8SJu4NsXSspF8/edit?usp=sharing 

bq. So...I would like to suggest to explore a different path and try to disable this feature in most of the modules, and identify and move the "bad" testcases to some integration module, where they can run isolated - after this someone may dig deeper into those bad cases and possibly fix them (and move them back 
Did not really consider this, since I just assumed that the tests were in a state where running them on a single node (as a single process) would take too long. As part of HIVE-13503, there's an effort to identify individual tests which run for a long time, and fix them. We should definitely explore running tests per-module and identify modules which can all be run as a single batch. Also 'unit' tests which run for far too long.


was (Author: sseth):
bq. AFAIK maven is currently configured to fork a jvm for every test(resuseforks=false)..
That was a little unexpected, or not... this needs to be fixed as well.

https://docs.google.com/spreadsheets/d/1QYLWp61kLWXEm6MGxwFrG5caTiTREm8SJu4NsXSspF8/edit?usp=sharing

In a ptest run reuseForks=false does not make a lot of difference (yet). Each individual test class is run via an independent mvn test invocation. That's over 600 mvn test invocations on 24 drones. The intent of batching was to amortize some of the time wasted in doing this. If we go down the batching route, reuseForks will come into play.

This sheet contains information on test runtimes - reported by junit vs reported by ptest.
https://docs.google.com/spreadsheets/d/1QYLWp61kLWXEm6MGxwFrG5caTiTREm8SJu4NsXSspF8/edit?usp=sharing 

bq. So...I would like to suggest to explore a different path and try to disable this feature in most of the modules, and identify and move the "bad" testcases to some integration module, where they can run isolated - after this someone may dig deeper into those bad cases and possibly fix them (and move them back 
Did not really consider this, since I just assumed that the tests were in a state where running them on a single node (as a single process) would take too long. As part of HIVE-13503, there's an effort to identify individual tests which run for a long time, and fix them. We should definitely explore running tests per-module and identify modules which can all be run as a single batch. Also 'unit' tests which run for far too long.

> Create batches for non qfile tests
> ----------------------------------
>
>                 Key: HIVE-14540
>                 URL: https://issues.apache.org/jira/browse/HIVE-14540
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Siddharth Seth
>            Assignee: Siddharth Seth
>
> From run 790:
> Reported runtime by junit: 17 hours
> Reported runtime by ptest: 34 hours
> A lot of time is wasted spinning up mvn test for each individual test, which otherwise takes less than 1 second. These tests could end up taking 20-30 seconds. Combined with HIVE-14539 - 60-70s.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)