You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cxf.apache.org by "gnodet (via GitHub)" <gi...@apache.org> on 2026/03/27 10:08:04 UTC

[PR] Improve tracing test diagnostics for flaky test investigation [cxf]

gnodet opened a new pull request, #2999:
URL: https://github.com/apache/cxf/pull/2999

   ## Summary
   
   - Replace `await().until(() -> spans.size() == N)` with `await().untilAsserted(() -> assertThat(message, size, equalTo(N)))` across all tracing system tests (Brave, OpenTelemetry, OpenTracing; JAX-RS and JAX-WS)
   - When the Awaitility wait times out, the error message now includes the actual span count **and** the full list of collected spans
   - This diagnostic improvement will help determine whether flaky failures (see #2971) are caused by: missing spans (count < expected), extra spans from cross-test contamination (count > expected), or a pure timing issue
   
   ## Context
   
   Related to #2971 and the discussion about root-causing the flaky tracing test failures. @reta pointed out that the thread-safety fix alone may not address all failures, since the OTel tests also fail despite using thread-safe `InMemorySpanExporter`. This PR adds diagnostic output so that when CI fails, we can see exactly what spans were collected and identify the root cause.
   
   ## Test plan
   
   - [x] Compilation verified locally (`mvn compile test-compile -pl systests/tracing`)
   - [ ] CI build should pass — no behavioral changes, only improved error messages on failure
   - [ ] When a tracing test times out in CI, the error message should now show the span list contents
   
   _Claude Code on behalf of Guillaume Nodet_
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@cxf.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Improve tracing test diagnostics for flaky test investigation [cxf]

Posted by "reta (via GitHub)" <gi...@apache.org>.
reta commented on code in PR #2999:
URL: https://github.com/apache/cxf/pull/2999#discussion_r3003762566


##########
systests/tracing/src/test/java/org/apache/cxf/systest/brave/jaxrs/AbstractBraveTracingTest.java:
##########
@@ -265,9 +265,10 @@ public void testThatProvidedSpanIsNotClosedWhenActive() throws MalformedURLExcep
         }
 
         // Await till flush happens, usually a second is enough
-        await().atMost(Duration.ofSeconds(5L)).until(()-> TestSpanHandler.getAllSpans().size() == 4);
+        await().atMost(Duration.ofSeconds(5L)).untilAsserted(() -> assertThat(

Review Comment:
   Thanks @gnodet , I think this is valuable improvement, do you think we could extract custom `Matcher` (for example, `HasSize`) out of it to have something like:
   
   ```
   await().atMost(Duration.ofSeconds(5L)).untilAsserted(() -> assertThat(TestSpanHandler.getAllSpans(), hasSize(4)));
   ```
   
   I could happily help with that, it will reduce the amount of duplication in assertions, thank you



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@cxf.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Improve tracing test diagnostics for flaky test investigation [cxf]

Posted by "reta (via GitHub)" <gi...@apache.org>.
reta merged PR #2999:
URL: https://github.com/apache/cxf/pull/2999


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@cxf.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Improve tracing test diagnostics for flaky test investigation [cxf]

Posted by "reta (via GitHub)" <gi...@apache.org>.
reta commented on code in PR #2999:
URL: https://github.com/apache/cxf/pull/2999#discussion_r3003762566


##########
systests/tracing/src/test/java/org/apache/cxf/systest/brave/jaxrs/AbstractBraveTracingTest.java:
##########
@@ -265,9 +265,10 @@ public void testThatProvidedSpanIsNotClosedWhenActive() throws MalformedURLExcep
         }
 
         // Await till flush happens, usually a second is enough
-        await().atMost(Duration.ofSeconds(5L)).until(()-> TestSpanHandler.getAllSpans().size() == 4);
+        await().atMost(Duration.ofSeconds(5L)).untilAsserted(() -> assertThat(

Review Comment:
   Thanks @gnodet , I think this is valuable improvement, do you think we could extract custom `Matcher` (for example, `HasSize` but for `Collection<MutableSpan>`) out of it to have something like:
   
   ```
   await().atMost(Duration.ofSeconds(5L)).untilAsserted(() -> assertThat(TestSpanHandler.getAllSpans(), hasSize(4)));
   ```
   
   I could happily help with that, it will reduce the amount of duplication in assertions, thank you



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@cxf.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Improve tracing test diagnostics for flaky test investigation [cxf]

Posted by "reta (via GitHub)" <gi...@apache.org>.
reta commented on code in PR #2999:
URL: https://github.com/apache/cxf/pull/2999#discussion_r3003762566


##########
systests/tracing/src/test/java/org/apache/cxf/systest/brave/jaxrs/AbstractBraveTracingTest.java:
##########
@@ -265,9 +265,10 @@ public void testThatProvidedSpanIsNotClosedWhenActive() throws MalformedURLExcep
         }
 
         // Await till flush happens, usually a second is enough
-        await().atMost(Duration.ofSeconds(5L)).until(()-> TestSpanHandler.getAllSpans().size() == 4);
+        await().atMost(Duration.ofSeconds(5L)).untilAsserted(() -> assertThat(

Review Comment:
   Thanks @gnodet , I think this is valuable improvement, do you think we could extract custom `Matcher` (for `HasSize`) out of it to have something like:
   
   ```
   await().atMost(Duration.ofSeconds(5L)).untilAsserted(() -> assertThat(TestSpanHandler.getAllSpans(), hasSize(4)));
   ```
   
   I could happily help with that, it will reduce the amount of duplication in assertions, thank you



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@cxf.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org