[jira] [Updated] (IMPALA-6070) Speed up test execution

["Tim Armstrong (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-6070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tim Armstrong updated IMPALA-6070:
----------------------------------
    Issue Type: Improvement  (was: Bug)

> Speed up test execution
> -----------------------
>
>                 Key: IMPALA-6070
>                 URL: https://issues.apache.org/jira/browse/IMPALA-6070
>             Project: IMPALA
>          Issue Type: Improvement
>            Reporter: Philip Zeyliger
>            Assignee: Philip Zeyliger
>            Priority: Major
>         Attachments: screenshot-1.png
>
>
> Our tests (e.g., https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/buildTimeTrend) tend to take about 4 hours. This can be improved.
> I'm opening this JIRA track those changes. I'm currently looking at:
> * Parallelizing multiple data-load steps: TPC-DS, TPC-H, and Functional take ~65 minutes when serialized. They take 35 minutes if running in parallel.
> * Parallelizing compute stats: this takes ~10 minutes; probably can be faster.
> The trickier thing is parallelizing fe tests, ee tests, and custom cluster tests. The approach I'm taking is to create a docker container with everything in it (including data load), and then running tests in parallel. This is a bit messier, but I think it has some legs when it comes to using machines with many cores.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org