You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by "gnodet (via GitHub)" <gi...@apache.org> on 2026/03/10 09:25:24 UTC
[PR] chore(ci): enable integration tests in incremental PR builds [camel]
gnodet opened a new pull request, #21899:
URL: https://github.com/apache/camel/pull/21899
## Summary
Remove `-DskipITs` from the incremental build script so that integration tests (failsafe) run alongside unit tests (surefire) during PR CI builds.
## Motivation
Currently, the incremental build in `pr-build-main.yml` always passes `-DskipITs`, which means integration tests are **never** exercised during PR CI. This creates a blind spot, particularly for:
- **Container version upgrade PRs** (e.g., bumping Ollama, Neo4j, Weaviate images in `test-infra/`): the whole point of these PRs is to verify compatibility with a new container version, but the ITs that spin up and test against these containers are skipped. These PRs are effectively merged untested.
- **Component changes** that may break integration behavior while unit tests still pass.
## Why this is safe
The incremental build script already has guards that prevent runaway CI:
| Scenario | What happens | Impact of this change |
|---|---|---|
| Root pom / `buildAll=true` | Tests are **completely skipped** (line 140: "Cannot launch the tests of all projects") | **None** — no tests run at all |
| Core module change (>50 dependents) | Only the directly changed modules are tested (no `-amd`) | ITs run on just the core module — typically very few or none |
| Typical component PR (<=50 dependents) | Changed modules + dependents tested with `-amd` | ITs run on the affected component and its dependents |
| >100 affected projects | Falls back to `buildAll=true` | **None** — no tests run |
The `maxNumberOfTestableProjects=50` cap ensures that ITs never run on more than ~50 modules, which keeps CI time bounded.
## Trade-offs
- **CI time may increase** for component PRs where ITs exist. Most components have 0-5 IT tests, so the overhead is typically small (a few minutes).
- **Flaky ITs** (e.g., container startup timeouts) may occasionally cause CI failures. However, this is valuable signal — if an IT is consistently flaky, it should be fixed or marked `@Disabled`.
## What changes
- `.github/actions/incremental-build/incremental-build.sh`: Remove `-DskipITs` from both test execution paths (direct modules and modules + dependents).
The `full` profile in `components/pom.xml` (active when `!quickly`) already binds failsafe's `integration-test` and `verify` goals to `mvn install`. Removing `-DskipITs` is all that's needed to enable them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore(ci): enable integration tests in incremental PR builds [camel]
Posted by "orpiske (via GitHub)" <gi...@apache.org>.
orpiske commented on PR #21899:
URL: https://github.com/apache/camel/pull/21899#issuecomment-4030006749
I also think it could be fine ... We are disabling specific tests on Github using `@DisabledIfSystemProperty(named = "ci.env.name", matches = ".*", disabledReason = "Flaky on Github CI")`, however, I don't know if we have maintained that list up-to-date. There may be some initial overhead, but I think it could be worth it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore(ci): enable integration tests in incremental PR builds [camel]
Posted by "gnodet (via GitHub)" <gi...@apache.org>.
gnodet commented on PR #21899:
URL: https://github.com/apache/camel/pull/21899#issuecomment-4029994992
@squakez Good point, thanks for raising it. I dug into the numbers:
- **1508** total IT files across `components/`
- **280** have `@Disabled` or `ci.env.name` guards → will be skipped in CI (all AI/LLM ITs that need real API keys or heavy containers fall here)
- **1228** without explicit guards, of which:
- **~114** directly reference `@RegisterExtension`/`@Container`/`Testcontainers` (many more inherit it from base classes like `FtpServerTestSupport`)
- Most use **embedded services** (embedded FTP, embedded Artemis, localstack for AWS, etc.) that work fine on GitHub Actions
The ITs that require "too much network resources" (Ollama, LangChain4j chat, kserve, etc.) are already guarded by `@DisabledIfSystemProperty(named = "ci.env.name", matches = ".*")`, and GH Actions sets `ci.env.name=github.com` via `MVND_OPTS`, so they'll remain skipped.
The ITs that **would** newly run are the ones using embedded services or local testcontainers (JMS/Artemis, Kafka, MongoDB, FTP, AWS localstack, etc.). These are designed to run in CI and many are already exercised by Jenkins. Enabling them on GH Actions too would catch regressions earlier in the PR cycle.
That said, if there are specific ITs that are known to be flaky or resource-heavy on GH Actions without proper `@Disabled` guards, we should fix those independently. This change would actually help surface them.
Happy to revert if this causes widespread CI failures, but I think the guards are already in place for the heavy tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore(ci): enable integration tests in incremental PR builds [camel]
Posted by "gnodet (via GitHub)" <gi...@apache.org>.
gnodet merged PR #21899:
URL: https://github.com/apache/camel/pull/21899
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore(ci): enable integration tests in incremental PR builds [camel]
Posted by "squakez (via GitHub)" <gi...@apache.org>.
squakez commented on PR #21899:
URL: https://github.com/apache/camel/pull/21899#issuecomment-4030028404
Yeah, I think it would be good to review at least all AI components which are the ones that surely cannot run on GH actions and mark them for not to run.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore(ci): enable integration tests in incremental PR builds [camel]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21899:
URL: https://github.com/apache/camel/pull/21899#issuecomment-4029947468
:star2: Thank you for your contribution to the Apache Camel project! :star2:
:robot: CI automation will test this PR automatically.
:camel: Apache Camel Committers, please review the following items:
* First-time contributors **require MANUAL approval** for the GitHub Actions to run
* You can use the command `/component-test (camel-)component-name1 (camel-)component-name2..` to request a test from the test bot although they are normally detected and executed by CI.
* You can label PRs using `build-all`, `build-dependents`, `skip-tests` and `test-dependents` to fine-tune the checks executed by this PR.
* Build and test logs are available in the summary page. **Only** [Apache Camel committers](https://camel.apache.org/community/team/#committers) have access to the summary.
:warning: Be careful when sharing logs. Review their contents before sharing them publicly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org