You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2022/03/23 06:31:05 UTC
[GitHub] [druid] paul-rogers edited a comment on issue #12359: Simplify and Accelerate Integration Tests
paul-rogers edited a comment on issue #12359:
URL: https://github.com/apache/druid/issues/12359#issuecomment-1075963969
@kfaraz, On the question about running tests in parallel: one reason we want to do that today is that every test builds Druid, builds the Docker image, and downloads a bunch of stuff. Even for something like the High Availability tests which then run in a few seconds. So, today, we're parallelizing wasted work. With the change, we'd build everything once, then run. Each run is:
* Start Docker Compose cluster using test-specific configs.
* Run the ITs for that group.
* Shut down the cluster.
If we find that is slow, then we can split the tests into two sets and do builds in both then run both.
There is still room for parallelism. The groups will run in sequence. However, it looks like we run with multiple configurations: with MySQL driver or with MariaDB driver. Secure or not secure, etc. Each of these variations must still run separately because we need to invoke the same maven modules, but with different profiles.
On Travis, one way to speed up more is if we could cache the Docker container and build artifacts (all the Druid jars). This means:
1. Build Druid and the Docker image. Place in a PR-specific repo.
2. Run all IT tests in parallel. Each pulls jars and the image from the above repo.
The above should work for config variations also, since the Druid jars and the test image are consistent; only the test code and "shared folder" would be twiddled to change configs.
I don't know Travis (and repos) well enough to know if this is possible, but it is something to consider.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org