You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by "gnodet (via GitHub)" <gi...@apache.org> on 2026/03/26 23:01:15 UTC
[PR] test-infra: Fix Docker container name collisions in parallel builds [camel]
gnodet opened a new pull request, #22287:
URL: https://github.com/apache/camel/pull/22287
## Summary
- Append JVM PID to Docker container names generated by `ContainerEnvironmentUtil.containerName()` to prevent `409 Conflict` errors when multiple modules sharing the same test infrastructure run integration tests in parallel (e.g., via mvnd)
## Problem
When mvnd tests multiple modules in parallel that share the same test infra service (e.g., `camel-elasticsearch` and `camel-elasticsearch-rest-client` both use `camel-test-infra-elasticsearch`), each JVM tries to create a Docker container with the same hardcoded name (e.g., `camel-elasticsearch`). The second JVM fails with:
```
ConflictException: Status 409: The container name "/camel-elasticsearch" is already in use
```
This causes all integration tests in the losing module to fail, as seen in #22237.
## Fix
Append `ProcessHandle.current().pid()` to the container name, making it unique per JVM (e.g., `camel-elasticsearch-12345`). This is safe because:
- The container name is only used for `cmd.withName()` (Docker identification), never for network aliases or inter-container communication
- Within a single JVM, the PID is constant so the `SingletonService` pattern still works
- Camel JBang `infra run/stop/ps` manages services by JVM PID, not Docker container names
- Also improves crash recovery: a zombie container from a crashed JVM no longer blocks the next run
## Test plan
- [x] `mvn install -B -pl test-infra/camel-test-infra-common -DskipTests` compiles successfully
- [ ] CI passes with parallel module testing
🤖 Generated with [Claude Code](https://claude.com/claude-code)
_Claude Code on behalf of Guillaume Nodet_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test-infra: Fix Docker container name collisions in parallel execution [camel]
Posted by "gnodet (via GitHub)" <gi...@apache.org>.
gnodet commented on PR #22287:
URL: https://github.com/apache/camel/pull/22287#issuecomment-4142835420
@Croway Good question! I looked into this — `camel infra run` manages services via PID files in `~/.camel/` (e.g., `infra-redis-<PID>.json`), not by container name. `camel infra stop`, `camel infra ps`, etc. all use this PID-based tracking. So the PID+counter suffix in the container name doesn't break any lifecycle management.
That said, I understand the UX concern: if someone runs `docker ps`, seeing `camel-kafka` is nicer than `camel-kafka-12345-1`.
Do you still think we should avoid the suffix specifically for the `camel infra run` use case? We could potentially use the `camel.infra.fixedPort` property as a signal — when it's `true` (set by JBang), skip the PID+counter and use the clean name, since in that mode only one instance runs at a time. Would that be doable?
_Claude Code on behalf of Guillaume Nodet_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test-infra: Fix Docker container name collisions in parallel builds [camel]
Posted by "gnodet (via GitHub)" <gi...@apache.org>.
gnodet commented on code in PR #22287:
URL: https://github.com/apache/camel/pull/22287#discussion_r2999970956
##########
test-infra/camel-test-infra-common/src/main/java/org/apache/camel/test/infra/common/services/ContainerEnvironmentUtil.java:
##########
@@ -114,6 +114,9 @@ public static String containerName(Class cls) {
if (annotation.serviceImplementationAlias().length > 0) {
name += "-" + annotation.serviceImplementationAlias()[0];
}
+ // Append PID to avoid Docker container name conflicts when multiple
+ // modules run tests in parallel (e.g., via mvnd with multiple threads)
+ name += "-" + ProcessHandle.current().pid();
Review Comment:
To be clear, the above analysis is not specific to Elasticsearch — `ContainerEnvironmentUtil.containerName()` is used by all ~30 test infra services (Kafka, MongoDB, Redis, etc.). The same collision would happen whenever two modules sharing the same test infra service are tested in parallel in separate JVMs. The PID suffix fix applies globally to all of them.
_Claude Code on behalf of Guillaume Nodet_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test-infra: Fix Docker container name collisions in parallel builds [camel]
Posted by "gnodet (via GitHub)" <gi...@apache.org>.
gnodet commented on code in PR #22287:
URL: https://github.com/apache/camel/pull/22287#discussion_r2999956105
##########
test-infra/camel-test-infra-common/src/main/java/org/apache/camel/test/infra/common/services/ContainerEnvironmentUtil.java:
##########
@@ -114,6 +114,9 @@ public static String containerName(Class cls) {
if (annotation.serviceImplementationAlias().length > 0) {
name += "-" + annotation.serviceImplementationAlias()[0];
}
+ // Append PID to avoid Docker container name conflicts when multiple
+ // modules run tests in parallel (e.g., via mvnd with multiple threads)
+ name += "-" + ProcessHandle.current().pid();
Review Comment:
To clarify further: the singleton is **per-JVM**, not cross-JVM.
**Within a JVM** (e.g., all test classes in `camel-elasticsearch`):
- `SingletonServiceHolder.INSTANCE` is a static field — one instance per classloader
- `store.computeIfAbsent("elastic", ...)` in JUnit's root `ExtensionContext.Store` ensures `initialize()` is called exactly once
- All test classes share the same container — same PID, same container name, no conflict
**Across JVMs** (e.g., `camel-elasticsearch` vs `camel-elasticsearch-rest-client` in separate mvnd daemons):
- Each JVM has its own static holder, its own JUnit root context
- Each creates its own Docker container independently
- The containers were **never shared** — without the PID fix, the second JVM simply crashed with a `409 Conflict`
So there was no actual container sharing happening across JVMs before either — it was just failing. The PID suffix makes it work by giving each JVM its own uniquely-named container.
_Claude Code on behalf of Guillaume Nodet_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test-infra: Fix Docker container name collisions in parallel execution [camel]
Posted by "gnodet (via GitHub)" <gi...@apache.org>.
gnodet commented on code in PR #22287:
URL: https://github.com/apache/camel/pull/22287#discussion_r3000325762
##########
test-infra/camel-test-infra-common/src/main/java/org/apache/camel/test/infra/common/services/ContainerEnvironmentUtil.java:
##########
@@ -114,6 +114,9 @@ public static String containerName(Class cls) {
if (annotation.serviceImplementationAlias().length > 0) {
name += "-" + annotation.serviceImplementationAlias()[0];
}
+ // Append PID to avoid Docker container name conflicts when multiple
+ // modules run tests in parallel (e.g., via mvnd with multiple threads)
+ name += "-" + ProcessHandle.current().pid();
Review Comment:
You were right that PID alone wasn't sufficient — there is indeed a within-JVM gap when test classes run in parallel. I've updated the fix to use PID + AtomicInteger counter, which covers both dimensions:
- **PID**: cross-JVM uniqueness (mvnd parallel builds)
- **Counter**: within-JVM uniqueness (parallel test classes)
Thread ID specifically wouldn't have worked (not unique across JVMs, and two services created on the same thread would still collide), but your instinct was correct. Thanks for pushing on this!
_Claude Code on behalf of Guillaume Nodet_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test-infra: Fix Docker container name collisions in parallel builds [camel]
Posted by "apupier (via GitHub)" <gi...@apache.org>.
apupier commented on code in PR #22287:
URL: https://github.com/apache/camel/pull/22287#discussion_r2999507089
##########
test-infra/camel-test-infra-common/src/main/java/org/apache/camel/test/infra/common/services/ContainerEnvironmentUtil.java:
##########
@@ -114,6 +114,9 @@ public static String containerName(Class cls) {
if (annotation.serviceImplementationAlias().length > 0) {
name += "-" + annotation.serviceImplementationAlias()[0];
}
+ // Append PID to avoid Docker container name conflicts when multiple
+ // modules run tests in parallel (e.g., via mvnd with multiple threads)
+ name += "-" + ProcessHandle.current().pid();
Review Comment:
shouldn't we use the thread id instead of the Process Id?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test-infra: Fix Docker container name collisions in parallel execution [camel]
Posted by "Croway (via GitHub)" <gi...@apache.org>.
Croway commented on PR #22287:
URL: https://github.com/apache/camel/pull/22287#issuecomment-4142879007
@gnodet picture the following blogpost example:
```
camel infra run postgres
docker exec -i camel-postgres psql -U test -d postgres < schema.sql
```
This scenario should keep working like this.
`We could potentially use the camel.infra.fixedPort property as a signal ` sounds hacky, but something like this should be done, yeah
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test-infra: Fix Docker container name collisions in parallel builds [camel]
Posted by "apupier (via GitHub)" <gi...@apache.org>.
apupier commented on code in PR #22287:
URL: https://github.com/apache/camel/pull/22287#discussion_r2999981371
##########
test-infra/camel-test-infra-common/src/main/java/org/apache/camel/test/infra/common/services/ContainerEnvironmentUtil.java:
##########
@@ -114,6 +114,9 @@ public static String containerName(Class cls) {
if (annotation.serviceImplementationAlias().length > 0) {
name += "-" + annotation.serviceImplementationAlias()[0];
}
+ // Append PID to avoid Docker container name conflicts when multiple
+ // modules run tests in parallel (e.g., via mvnd with multiple threads)
+ name += "-" + ProcessHandle.current().pid();
Review Comment:
ok, then the comment is a bit misleading as it mentions multiple threads
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test-infra: Fix Docker container name collisions in parallel execution [camel]
Posted by "gnodet (via GitHub)" <gi...@apache.org>.
gnodet commented on PR #22287:
URL: https://github.com/apache/camel/pull/22287#issuecomment-4142961982
@Croway Done — pushed a new commit that skips the PID+counter suffix when `camel.infra.fixedPort=true` (set by `camel infra run`). So `docker exec -i camel-postgres psql ...` keeps working as expected.
In test mode (default), the unique suffix is still appended for parallel safety.
_Claude Code on behalf of Guillaume Nodet_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test-infra: Fix Docker container name collisions in parallel execution [camel]
Posted by "gnodet (via GitHub)" <gi...@apache.org>.
gnodet merged PR #22287:
URL: https://github.com/apache/camel/pull/22287
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test-infra: Fix Docker container name collisions in parallel builds [camel]
Posted by "gnodet (via GitHub)" <gi...@apache.org>.
gnodet commented on code in PR #22287:
URL: https://github.com/apache/camel/pull/22287#discussion_r2999966433
##########
test-infra/camel-test-infra-common/src/main/java/org/apache/camel/test/infra/common/services/ContainerEnvironmentUtil.java:
##########
@@ -114,6 +114,9 @@ public static String containerName(Class cls) {
if (annotation.serviceImplementationAlias().length > 0) {
name += "-" + annotation.serviceImplementationAlias()[0];
}
+ // Append PID to avoid Docker container name conflicts when multiple
+ // modules run tests in parallel (e.g., via mvnd with multiple threads)
+ name += "-" + ProcessHandle.current().pid();
Review Comment:
Also worth noting: the tests are **designed to share a single container** within a module. Test isolation is not achieved through container isolation but through **data partitioning** — each test method gets a unique Elasticsearch index prefix via `testNameExtension.getCurrentTestName()` (see `ElasticsearchTestSupport.createPrefix()`). This avoids the overhead of restarting Elasticsearch for each test class.
The two modules (`camel-elasticsearch` and `camel-elasticsearch-rest-client`) were never sharing a container either — they run in separate surefire/failsafe forks (separate JVMs), each with its own `SingletonServiceHolder.INSTANCE`. The second JVM was simply crashing trying to create a container with an already-taken Docker name. The PID suffix makes both succeed with their own container, which is what was always intended.
_Claude Code on behalf of Guillaume Nodet_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test-infra: Fix Docker container name collisions in parallel builds [camel]
Posted by "gnodet (via GitHub)" <gi...@apache.org>.
gnodet commented on PR #22287:
URL: https://github.com/apache/camel/pull/22287#issuecomment-4141514528
### Follow-up: making integration tests more robust for parallel execution
This PR fixes the **cross-JVM** container name collision (e.g., mvnd testing two modules in parallel). However, there's a broader gap: **within-JVM** parallel execution of test classes using non-singleton services is only safe because `camel.failsafe.parallel` defaults to `false`.
If parallel integration test execution is ever enabled, non-singleton services sharing the same `@InfraService(serviceAlias = ...)` would collide on the Docker container name within the same JVM — and the PID suffix doesn't help there since all threads share the same PID.
JUnit 5's `@ResourceLock` mechanism could address this by declaring the Docker container name as a shared resource, preventing concurrent test classes from conflicting. We should consider a follow-up to:
- Have test infra services declare `@ResourceLock` on the container name
- Or migrate non-singleton services to the singleton pattern where appropriate
This would make the test infrastructure safe for parallel execution without relying on convention.
_Claude Code on behalf of Guillaume Nodet_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test-infra: Fix Docker container name collisions in parallel builds [camel]
Posted by "gnodet (via GitHub)" <gi...@apache.org>.
gnodet commented on code in PR #22287:
URL: https://github.com/apache/camel/pull/22287#discussion_r2999943447
##########
test-infra/camel-test-infra-common/src/main/java/org/apache/camel/test/infra/common/services/ContainerEnvironmentUtil.java:
##########
@@ -114,6 +114,9 @@ public static String containerName(Class cls) {
if (annotation.serviceImplementationAlias().length > 0) {
name += "-" + annotation.serviceImplementationAlias()[0];
}
+ // Append PID to avoid Docker container name conflicts when multiple
+ // modules run tests in parallel (e.g., via mvnd with multiple threads)
+ name += "-" + ProcessHandle.current().pid();
Review Comment:
The conflict happens between **separate JVM processes** (mvnd spawns each module in its own daemon JVM), not between threads within the same JVM. Thread ID wouldn't help because:
1. Thread IDs are not unique across processes — e.g., both JVMs' main threads could have thread ID 1
2. Within a single JVM, the `SingletonService` pattern already ensures only one container is created per service, shared across all test classes/threads
3. Thread ID can vary depending on which thread first triggers the container creation, making the name non-deterministic within the same JVM — which could actually *cause* collisions where none exist today
PID is unique per JVM on the host and stable for the entire JVM lifetime, which is exactly the granularity we need.
_Claude Code on behalf of Guillaume Nodet_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test-infra: Fix Docker container name collisions in parallel execution [camel]
Posted by "Croway (via GitHub)" <gi...@apache.org>.
Croway commented on PR #22287:
URL: https://github.com/apache/camel/pull/22287#issuecomment-4141842185
@gnodet is the PID appended when using `camel infra run $SERVICE` too? in this case shouldn't be.. when using `camel infra run ..` it is handy to have the same name so that it can be used in guides, documentation, and development
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test-infra: Fix Docker container name collisions in parallel builds [camel]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22287:
URL: https://github.com/apache/camel/pull/22287#issuecomment-4138877669
:star2: Thank you for your contribution to the Apache Camel project! :star2:
:robot: CI automation will test this PR automatically.
:camel: Apache Camel Committers, please review the following items:
* First-time contributors **require MANUAL approval** for the GitHub Actions to run
* You can use the command `/component-test (camel-)component-name1 (camel-)component-name2..` to request a test from the test bot although they are normally detected and executed by CI.
* You can label PRs using `build-all`, `build-dependents`, `skip-tests` and `test-dependents` to fine-tune the checks executed by this PR.
* Build and test logs are available in the summary page. **Only** [Apache Camel committers](https://camel.apache.org/community/team/#committers) have access to the summary.
:warning: Be careful when sharing logs. Review their contents before sharing them publicly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org