You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by GitBox <gi...@apache.org> on 2022/05/24 08:27:08 UTC
[GitHub] [camel-k] tadayosi opened a new issue, #3298: Our preference on defining multiple e2e tests
tadayosi opened a new issue, #3298:
URL: https://github.com/apache/camel-k/issues/3298
Our e2e testing is growing and we already have many e2e tests. We also see the time of e2e test runs in CI growing along with features we add to the codebase.
So what is our preferable style to define multiple flavours of e2e tests on the same target function? For example, we want to test the cli command `kamel run` in multiple scenarios such as running with different locations of sources, or with different combinations of options, etc.
https://github.com/apache/camel-k/blob/main/e2e/common/cli/run_test.go
There are two ways of defining those different scenarios (flavours):
1. Have multiple distinct `TestRun_XXX()` functions, with each running on a different namespace and integration platform installation.
2. Have a single `TestRun()` function and define multiple `t.Run("xxx", ...)` blocks inside `WithNewTestNamespace()`, which uses only a single namespace and platform throughout the test suite.
Right now, we seem to mix the two styles without any explicit policies. But possibly we should prefer one over another.
What I'm thinking now is that perhaps we should prefer the option #2 throughout the e2e testing by default. It's because we only create and destroy one namespace and platform for each collection of tests and it might make our e2e testing more efficient and fast. One potential drawback on the option #2 is that it might introduce hidden dependencies to each test in the same collection so we might need to be careful about leftovers in each test. But I naively think it's rare to happen.
WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-k] squakez commented on issue #3298: Our preference on defining multiple e2e tests
Posted by GitBox <gi...@apache.org>.
squakez commented on issue #3298:
URL: https://github.com/apache/camel-k/issues/3298#issuecomment-1135578276
Hey @tadayosi I'm always using the 2nd approach. As you've pointed out correctly, installing the operator for every test is seldom necessary. I guess there may be special situations where we need to create a new installation, but I'd say we should prefer option 2 when we can. About the single local testing, it's true, it may be a bit bothering, but I use to comment the code I don't need to test as a quick way to launch a single test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-k] tadayosi closed issue #3298: Our preference on defining multiple e2e tests
Posted by GitBox <gi...@apache.org>.
tadayosi closed issue #3298: Our preference on defining multiple e2e tests
URL: https://github.com/apache/camel-k/issues/3298
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-k] tadayosi commented on issue #3298: Our preference on defining multiple e2e tests
Posted by GitBox <gi...@apache.org>.
tadayosi commented on issue #3298:
URL: https://github.com/apache/camel-k/issues/3298#issuecomment-1135576890
Another drawback in the option 2 is that you cannot run a single test in the collection as you can in the option 1. In the option 1, you can run a single test by using `go test ... -run TestRun_XXX` option.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-k] tadayosi commented on issue #3298: Our preference on defining multiple e2e tests
Posted by GitBox <gi...@apache.org>.
tadayosi commented on issue #3298:
URL: https://github.com/apache/camel-k/issues/3298#issuecomment-1143273386
OK, most of the e2e tests follow the 2nd approach already. The remaining tests that don't follow it are rather exceptional.
Let's then use this issue to rewrite the remaining to align with the 2nd approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org