You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "santhoshct (via GitHub)" <gi...@apache.org> on 2024/12/16 13:13:35 UTC
[PR] flaky test report improvements [kafka]
santhoshct opened a new pull request, #18212:
URL: https://github.com/apache/kafka/pull/18212
- Added test cases to cleared test reports
- Added table view to all the remaining reports
### Testing
# Flaky Test Report for 2024-12-16
This report was run on 2024-12-16 13:01:46 UTC
## Summary of Most Problematic Tests
Found 7 tests that have been quarantined for a while and are still flaky.
<table><tr><td>Class</td><td>Test Case</td><td>Failure Rate</td><td>Build Scans</td></tr>
<tr><td colspan="4">org.apache.kafka.tiered.storage.integration.OffloadAndTxnConsumeFromLeaderTest</td></tr>
<tr><td></td><td>executeTieredStorageTest(String, String)[1] </td><td>100.00%</td><td>2</td></tr>
<tr><td></td><td>executeTieredStorageTest(String, String)[2] </td><td>100.00%</td><td>2</td></tr>
<tr><td colspan="4">org.apache.kafka.clients.consumer.internals.AbstractCoordinatorTest</td></tr>
<tr><td></td><td>testWakeupAfterSyncGroupReceived() </td><td>25.00%</td><td>4</td></tr>
<tr><td colspan="4">org.apache.kafka.common.network.Tls13SelectorTest</td></tr>
<tr><td></td><td>testCloseOldestConnection() </td><td>25.00%</td><td>4</td></tr>
<tr><td colspan="4">org.apache.kafka.common.network.SslTransportLayerTest</td></tr>
<tr><td></td><td>testSelectorPollReadSize(Args)[1] </td><td>25.00%</td><td>4</td></tr>
<tr><td colspan="4">org.apache.kafka.tiered.storage.integration.TransactionsWithTieredStoreTest</td></tr>
<tr><td></td><td>testReadCommittedConsumerShouldNotSeeUndecidedData(String, String)[2]</td><td>17.65%</td><td>578</td></tr>
<tr><td></td><td>testBumpTransactionalEpochWithTV2Enabled(String, String, boolean)[2]</td><td>15.92%</td><td>578</td></tr>
<tr><td></td><td>testBumpTransactionalEpochWithTV2Enabled(String, String, boolean)[1]</td><td>15.40%</td><td>578</td></tr>
<tr><td></td><td>testBumpTransactionalEpochWithTV2Disabled(String, String, boolean)[1]</td><td>9.00%</td><td>578</td></tr>
<tr><td></td><td>testBumpTransactionalEpochWithTV2Disabled(String, String, boolean)[2]</td><td>4.84%</td><td>578</td></tr>
<tr><td colspan="4">kafka.api.TransactionsTest</td></tr>
<tr><td></td><td>testBumpTransactionalEpochWithTV2Enabled(String, String, boolean)[1]</td><td>13.19%</td><td>576</td></tr>
<tr><td></td><td>testBumpTransactionalEpochWithTV2Enabled(String, String, boolean)[2]</td><td>11.11%</td><td>576</td></tr>
<tr><td></td><td>testBumpTransactionalEpochWithTV2Disabled(String, String, boolean)[1]</td><td>7.12%</td><td>576</td></tr>
<tr><td></td><td>testBumpTransactionalEpochWithTV2Disabled(String, String, boolean)[2]</td><td>5.03%</td><td>576</td></tr>
<tr><td colspan="4">kafka.api.PlaintextConsumerTest</td></tr>
<tr><td></td><td>testCloseLeavesGroupOnInterrupt(String, String)[2] </td><td>11.25%</td><td>578</td></tr>
<tr><td></td><td>testCoordinatorFailover(String, String)[2] </td><td>3.11%</td><td>578</td></tr>
<tr><td></td><td>testCloseLeavesGroupOnInterrupt(String, String)[1] </td><td>2.60%</td><td>578</td></tr>
<tr><td></td><td>testCoordinatorFailover(String, String)[1] </td><td>2.42%</td><td>578</td></tr>
</table>
## Flaky Test Regressions
No flaky test regressions found.
## Cleared Tests (Ready for Unquarantine)
<table>
<tr><td>Test Class</td><td>Test Method</td><td>Success Rate</td><td>Total Runs</td><td>Recent Status</td></tr>
<tr><td colspan="5">org.apache.kafka.streams.AutoOffsetResetTest</td></tr>
<tr><td></td><td>Class Overall</td><td>99.40%</td><td>336</td><td>334 passed</td></tr>
<tr><td></td><td>shouldThrowExceptionOnDurationForEarliestReset()</td><td>99.40%</td><td>334</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td></td><td>shouldThrowExceptionOnDurationForLastetReset()</td><td>99.40%</td><td>334</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td></td><td>shouldThrowExceptionOnDurationForNoneReset()</td><td>99.40%</td><td>334</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">org.apache.kafka.controller.QuorumControllerTest</td></tr>
<tr><td></td><td>Class Overall</td><td>98.83%</td><td>343</td><td>339 passed</td></tr>
<tr><td></td><td>testUncleanShutdownElrDisabled()</td><td>99.71%</td><td>347</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-14 03:24 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-14 03:24 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-14 14:27 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-15 01:15 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-15 01:15 - passed</td></tr>
<tr><td></td><td>testUncleanShutdownBrokerElrEnabled()</td><td>99.14%</td><td>347</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-14 03:24 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-14 03:24 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-14 14:27 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-15 01:15 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-15 01:15 - passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">kafka.api.PlaintextConsumerSubscriptionTest</td></tr>
<tr><td></td><td>Class Overall</td><td>99.78%</td><td>465</td><td>464 passed</td></tr>
<tr><td></td><td>testRe2JPatternExpandSubscription(String, String)[1]</td><td>99.80%</td><td>495</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">org.apache.kafka.streams.integration.FineGrainedAutoResetIntegrationTest</td></tr>
<tr><td></td><td>Class Overall</td><td>97.42%</td><td>310</td><td>302 passed</td></tr>
<tr><td></td><td>shouldResetByDuration()</td><td>98.81%</td><td>336</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td></td><td>shouldFailForResetNone()</td><td>98.81%</td><td>336</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">org.apache.kafka.streams.StreamsBuilderTest</td></tr>
<tr><td></td><td>Class Overall</td><td>99.27%</td><td>548</td><td>544 passed</td></tr>
<tr><td></td><td>shouldWrapProcessorsForStreamStreamSelfJoinWithSharedStoreOptimization()</td><td>99.20%</td><td>251</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">kafka.api.SaslPlaintextConsumerTest</td></tr>
<tr><td></td><td>Class Overall</td><td>96.16%</td><td>547</td><td>526 passed</td></tr>
<tr><td></td><td>testCoordinatorFailover(String, String)[1]</td><td>99.83%</td><td>577</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td></td><td>testCoordinatorFailover(String, String)[2]</td><td>96.19%</td><td>577</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">org.apache.kafka.clients.producer.internals.TransactionManagerTest</td></tr>
<tr><td></td><td>Class Overall</td><td>99.53%</td><td>422</td><td>420 passed</td></tr>
<tr><td></td><td>testTransactionV2AddPartitionAndOffsets()</td><td>99.53%</td><td>428</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-14 03:24 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-14 03:24 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-14 14:27 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-15 01:15 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-15 01:15 - passed</td></tr>
<tr><td></td><td>testMaybeAddPartitionToTransactionInTransactionV2()</td><td>99.53%</td><td>428</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-14 03:24 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-14 03:24 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-14 14:27 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-15 01:15 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-15 01:15 - passed</td></tr>
<tr><td></td><td>testFatalErrorWhenProduceResponseWithInvalidPidMapping()</td><td>99.53%</td><td>428</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-14 03:24 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-14 03:24 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-14 14:27 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-15 01:15 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-15 01:15 - passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">org.apache.kafka.clients.consumer.internals.ApplicationEventHandlerTest</td></tr>
<tr><td></td><td>Class Overall</td><td>98.29%</td><td>175</td><td>172 passed</td></tr>
<tr><td></td><td>testRecordApplicationEventQueueSize()</td><td>98.49%</td><td>199</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">org.apache.kafka.streams.state.internals.metrics.RocksDBMetricsRecorderTest</td></tr>
<tr><td></td><td>Class Overall</td><td>97.85%</td><td>93</td><td>91 passed</td></tr>
<tr><td></td><td>shouldThrowIfMetricRecorderIsInitialisedWithNullTaskId()</td><td>98.29%</td><td>117</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td></td><td>shouldThrowIfMetricRecorderIsInitialisedWithNullMetrics()</td><td>98.29%</td><td>117</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">org.apache.kafka.streams.kstream.internals.KStreamImplTest</td></tr>
<tr><td></td><td>Class Overall</td><td>99.37%</td><td>316</td><td>314 passed</td></tr>
<tr><td></td><td>shouldProcessWithProcessorAndState()</td><td>99.40%</td><td>334</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">kafka.network.DynamicConnectionQuotaTest</td></tr>
<tr><td></td><td>Class Overall</td><td>95.60%</td><td>546</td><td>522 passed</td></tr>
<tr><td></td><td>testDynamicConnectionQuota(String)[1]</td><td>95.83%</td><td>576</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">kafka.api.SslConsumerTest</td></tr>
<tr><td></td><td>Class Overall</td><td>99.45%</td><td>546</td><td>543 passed</td></tr>
<tr><td></td><td>testCoordinatorFailover(String, String)[1]</td><td>99.83%</td><td>576</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td></td><td>testCoordinatorFailover(String, String)[2]</td><td>99.65%</td><td>576</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">kafka.api.SaslSslConsumerTest</td></tr>
<tr><td></td><td>Class Overall</td><td>99.09%</td><td>548</td><td>543 passed</td></tr>
<tr><td></td><td>testCoordinatorFailover(String, String)[1]</td><td>99.13%</td><td>578</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td></td><td>testCoordinatorFailover(String, String)[2]</td><td>99.83%</td><td>578</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">org.apache.kafka.connect.integration.OffsetsApiIntegrationTest</td></tr>
<tr><td></td><td>Class Overall</td><td>93.25%</td><td>548</td><td>511 passed</td></tr>
<tr><td></td><td>testResetSinkConnectorOffsets()</td><td>98.27%</td><td>578</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td></td><td>testResetSinkConnectorOffsetsOverriddenConsumerGroupId()</td><td>97.92%</td><td>578</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td></td><td>testAlterSinkConnectorOffsetsOverriddenConsumerGroupId()</td><td>99.31%</td><td>578</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td></td><td>testGetSinkConnectorOffsetsDifferentKafkaClusterTargeted()</td><td>97.40%</td><td>578</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td></td><td>testAlterSinkConnectorOffsetsDifferentKafkaClusterTargeted()</td><td>96.02%</td><td>578</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">kafka.test.api.ShareConsumerTest</td></tr>
<tr><td></td><td>Class Overall</td><td>98.53%</td><td>546</td><td>538 passed</td></tr>
<tr><td></td><td>testAcknowledgementCommitCallbackCallsShareConsumerWakeup(String)[1]</td><td>99.83%</td><td>576</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td></td><td>testAcknowledgementCommitCallbackCallsShareConsumerWakeup(String)[2]</td><td>99.65%</td><td>576</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td></td><td>testAcknowledgementCommitCallbackInvalidRecordStateException(String)[1]</td><td>99.31%</td><td>576</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td></td><td>testMultipleConsumersInGroupConcurrentConsumption(String)[1]</td><td>99.83%</td><td>576</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td></td><td>testExplicitAcknowledgementCommitAsync(String)[1]</td><td>99.83%</td><td>576</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">kafka.api.SaslPlainPlaintextConsumerTest</td></tr>
<tr><td></td><td>Class Overall</td><td>98.18%</td><td>548</td><td>538 passed</td></tr>
<tr><td></td><td>testCoordinatorFailover(String, String)[1]</td><td>99.65%</td><td>578</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td></td><td>testCoordinatorFailover(String, String)[2]</td><td>98.62%</td><td>578</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">kafka.api.SaslMultiMechanismConsumerTest</td></tr>
<tr><td></td><td>Class Overall</td><td>97.26%</td><td>548</td><td>533 passed</td></tr>
<tr><td></td><td>testCoordinatorFailover(String, String)[1]</td><td>98.44%</td><td>578</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td></td><td>testCoordinatorFailover(String, String)[2]</td><td>98.79%</td><td>578</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">kafka.api.SslAdminIntegrationTest</td></tr>
<tr><td></td><td>Class Overall</td><td>99.51%</td><td>205</td><td>204 passed</td></tr>
<tr><td></td><td>testListNodesFromControllersIncludingFencedBrokers(String)[1]</td><td>99.56%</td><td>227</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">kafka.api.PlaintextAdminIntegrationTest</td></tr>
<tr><td></td><td>Class Overall</td><td>99.58%</td><td>475</td><td>473 passed</td></tr>
<tr><td></td><td>testConsumerGroupWithMemberMigration(String)[1]</td><td>99.60%</td><td>501</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">kafka.api.PlaintextConsumerTest</td></tr>
<tr><td></td><td>Class Overall</td><td>82.66%</td><td>548</td><td>453 passed</td></tr>
<tr><td></td><td>testCoordinatorFailover(String, String)[2]</td><td>96.89%</td><td>578</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td></td><td>testCoordinatorFailover(String, String)[1]</td><td>97.58%</td><td>578</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td></td><td>testCloseLeavesGroupOnInterrupt(String, String)[1]</td><td>97.40%</td><td>578</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td></td><td>testCloseLeavesGroupOnInterrupt(String, String)[2]</td><td>88.75%</td><td>578</td><td>passed</td></tr>
<tr><td></td><td colspan="4">Recent Executions:</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:46 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 06:48 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:28 - passed</td></tr>
<tr><td></td><td colspan="4">2024-12-16 11:29 - passed</td></tr>
<tr><td colspan="5"> </td></tr>
</table>
## High-Priority Quarantined Tests
These are tests which have been quarantined for several days and need attention.
Found 7 high-priority quarantined test classes:
<table>
<tr><td>Test Class</td><td>Days Quarantined</td><td>Failure Rate</td><td>Recent Failure Rate</td><td>Total Runs</td></tr>
<tr><td colspan="5">org.apache.kafka.tiered.storage.integration.OffloadAndTxnConsumeFromLeaderTest</td></tr>
<tr><td></td><td>7</td><td>100.00%</td><td>100.00%</td><td>2</td></tr>
<tr><td colspan="5">Build Outcomes:</td></tr>
<tr><td></td><td colspan="4">Passed: 0 | Failed: 2 | Flaky: 0</td></tr>
<tr><td colspan="5">Test Methods (Last 7 Days):</td></tr>
<tr><td>Method</td><td>Success Rate</td><td>Failure Rate</td><td>Total Runs</td><td>Recent Status</td></tr>
<tr><td>executeTieredStorageTest(String, String)[1]</td><td>0.00%</td><td>100.00%</td><td>2</td><td>flaky</td></tr>
<tr><td>executeTieredStorageTest(String, String)[2]</td><td>0.00%</td><td>100.00%</td><td>2</td><td>flaky</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">org.apache.kafka.tiered.storage.integration.TransactionsWithTieredStoreTest</td></tr>
<tr><td></td><td>7</td><td>46.37%</td><td>45.80%</td><td>578</td></tr>
<tr><td colspan="5">Build Outcomes:</td></tr>
<tr><td></td><td colspan="4">Passed: 310 | Failed: 8 | Flaky: 260</td></tr>
<tr><td colspan="5">Test Methods (Last 7 Days):</td></tr>
<tr><td>Method</td><td>Success Rate</td><td>Failure Rate</td><td>Total Runs</td><td>Recent Status</td></tr>
<tr><td>testReadCommittedConsumerShouldNotSeeUndecidedData(String, String)[2]</td><td>82.35%</td><td>17.65%</td><td>578</td><td>passed</td></tr>
<tr><td>testBumpTransactionalEpochWithTV2Enabled(String, String, boolean)[2]</td><td>84.08%</td><td>15.92%</td><td>578</td><td>passed</td></tr>
<tr><td>testBumpTransactionalEpochWithTV2Enabled(String, String, boolean)[1]</td><td>84.60%</td><td>15.40%</td><td>578</td><td>flaky</td></tr>
<tr><td>testBumpTransactionalEpochWithTV2Disabled(String, String, boolean)[1]</td><td>91.00%</td><td>9.00%</td><td>578</td><td>passed</td></tr>
<tr><td>testBumpTransactionalEpochWithTV2Disabled(String, String, boolean)[2]</td><td>95.16%</td><td>4.84%</td><td>578</td><td>flaky</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">kafka.api.TransactionsTest</td></tr>
<tr><td></td><td>7</td><td>28.65%</td><td>29.30%</td><td>576</td></tr>
<tr><td colspan="5">Build Outcomes:</td></tr>
<tr><td></td><td colspan="4">Passed: 411 | Failed: 9 | Flaky: 156</td></tr>
<tr><td colspan="5">Test Methods (Last 7 Days):</td></tr>
<tr><td>Method</td><td>Success Rate</td><td>Failure Rate</td><td>Total Runs</td><td>Recent Status</td></tr>
<tr><td>testBumpTransactionalEpochWithTV2Enabled(String, String, boolean)[1]</td><td>86.81%</td><td>13.19%</td><td>576</td><td>passed</td></tr>
<tr><td>testBumpTransactionalEpochWithTV2Enabled(String, String, boolean)[2]</td><td>88.89%</td><td>11.11%</td><td>576</td><td>passed</td></tr>
<tr><td>testBumpTransactionalEpochWithTV2Disabled(String, String, boolean)[1]</td><td>92.88%</td><td>7.12%</td><td>576</td><td>passed</td></tr>
<tr><td>testBumpTransactionalEpochWithTV2Disabled(String, String, boolean)[2]</td><td>94.97%</td><td>5.03%</td><td>576</td><td>passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">org.apache.kafka.clients.consumer.internals.AbstractCoordinatorTest</td></tr>
<tr><td></td><td>7</td><td>25.00%</td><td>25.00%</td><td>4</td></tr>
<tr><td colspan="5">Build Outcomes:</td></tr>
<tr><td></td><td colspan="4">Passed: 3 | Failed: 0 | Flaky: 1</td></tr>
<tr><td colspan="5">Test Methods (Last 7 Days):</td></tr>
<tr><td>Method</td><td>Success Rate</td><td>Failure Rate</td><td>Total Runs</td><td>Recent Status</td></tr>
<tr><td>testWakeupAfterSyncGroupReceived()</td><td>75.00%</td><td>25.00%</td><td>4</td><td>passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">org.apache.kafka.common.network.Tls13SelectorTest</td></tr>
<tr><td></td><td>7</td><td>25.00%</td><td>25.00%</td><td>4</td></tr>
<tr><td colspan="5">Build Outcomes:</td></tr>
<tr><td></td><td colspan="4">Passed: 3 | Failed: 0 | Flaky: 1</td></tr>
<tr><td colspan="5">Test Methods (Last 7 Days):</td></tr>
<tr><td>Method</td><td>Success Rate</td><td>Failure Rate</td><td>Total Runs</td><td>Recent Status</td></tr>
<tr><td>testCloseOldestConnection()</td><td>75.00%</td><td>25.00%</td><td>4</td><td>passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">org.apache.kafka.common.network.SslTransportLayerTest</td></tr>
<tr><td></td><td>7</td><td>25.00%</td><td>25.00%</td><td>4</td></tr>
<tr><td colspan="5">Build Outcomes:</td></tr>
<tr><td></td><td colspan="4">Passed: 3 | Failed: 0 | Flaky: 1</td></tr>
<tr><td colspan="5">Test Methods (Last 7 Days):</td></tr>
<tr><td>Method</td><td>Success Rate</td><td>Failure Rate</td><td>Total Runs</td><td>Recent Status</td></tr>
<tr><td>testSelectorPollReadSize(Args)[1]</td><td>75.00%</td><td>25.00%</td><td>4</td><td>passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">kafka.api.PlaintextConsumerTest</td></tr>
<tr><td></td><td>7</td><td>17.13%</td><td>17.34%</td><td>578</td></tr>
<tr><td colspan="5">Build Outcomes:</td></tr>
<tr><td></td><td colspan="4">Passed: 479 | Failed: 0 | Flaky: 99</td></tr>
<tr><td colspan="5">Test Methods (Last 7 Days):</td></tr>
<tr><td>Method</td><td>Success Rate</td><td>Failure Rate</td><td>Total Runs</td><td>Recent Status</td></tr>
<tr><td>testCloseLeavesGroupOnInterrupt(String, String)[2]</td><td>88.75%</td><td>11.25%</td><td>578</td><td>passed</td></tr>
<tr><td>testCoordinatorFailover(String, String)[2]</td><td>96.89%</td><td>3.11%</td><td>578</td><td>passed</td></tr>
<tr><td>testCloseLeavesGroupOnInterrupt(String, String)[1]</td><td>97.40%</td><td>2.60%</td><td>578</td><td>passed</td></tr>
<tr><td>testCoordinatorFailover(String, String)[1]</td><td>97.58%</td><td>2.42%</td><td>578</td><td>passed</td></tr>
<tr><td colspan="5"> </td></tr>
</table>
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] flaky test report improvements [kafka]
Posted by "mumrah (via GitHub)" <gi...@apache.org>.
mumrah commented on PR #18212:
URL: https://github.com/apache/kafka/pull/18212#issuecomment-2545631533
@santhoshct some initial notes:
1) In Cleared Tests, just include the test class and test case lines.
2) See if we can move the detailed info (with recent builds) into a collapsible section (`<summary><detail>` or something?)
3) Add a link to Develocity for test classes and test cases.
Thanks for continuing work on this!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] flaky test report improvements [kafka]
Posted by "santhoshct (via GitHub)" <gi...@apache.org>.
santhoshct commented on PR #18212:
URL: https://github.com/apache/kafka/pull/18212#issuecomment-2546175774
All three successfully done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] flaky test report improvements [kafka]
Posted by "santhoshct (via GitHub)" <gi...@apache.org>.
santhoshct commented on PR #18212:
URL: https://github.com/apache/kafka/pull/18212#issuecomment-2546007583
<table>
<tr><td>Test Class</td><td>Days Quarantined</td><td>Failure Rate</td><td>Recent Failure Rate</td><td>Total Runs</td></tr>
<tr><td colspan="5">org.apache.kafka.tiered.storage.integration.OffloadAndTxnConsumeFromLeaderTest</td></tr>
<tr><td></td><td>7</td><td>100.00%</td><td>100.00%</td><td>2</td></tr>
<tr><td colspan="5">Build Outcomes: Passed: 0 | Failed: 2 | Flaky: 0</td></tr>
<tr><td colspan="5">Test Methods (Last 7 Days):</td></tr>
<tr><td>Method</td><td>Success Rate</td><td>Failure Rate</td><td>Total Runs</td><td>Recent Status</td></tr>
<tr><td>executeTieredStorageTest(String, String)[1]</td><td>0.00%</td><td>100.00%</td><td>2</td><td>flaky</td></tr>
<tr><td>executeTieredStorageTest(String, String)[2]</td><td>0.00%</td><td>100.00%</td><td>2</td><td>flaky</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">org.apache.kafka.tiered.storage.integration.TransactionsWithTieredStoreTest</td></tr>
<tr><td></td><td>7</td><td>45.88%</td><td>45.35%</td><td>582</td></tr>
<tr><td colspan="5">Build Outcomes: Passed: 315 | Failed: 8 | Flaky: 259</td></tr>
<tr><td colspan="5">Test Methods (Last 7 Days):</td></tr>
<tr><td>Method</td><td>Success Rate</td><td>Failure Rate</td><td>Total Runs</td><td>Recent Status</td></tr>
<tr><td>testReadCommittedConsumerShouldNotSeeUndecidedData(String, String)[2]</td><td>82.65%</td><td>17.35%</td><td>582</td><td>flaky</td></tr>
<tr><td>testBumpTransactionalEpochWithTV2Enabled(String, String, boolean)[2]</td><td>84.36%</td><td>15.64%</td><td>582</td><td>passed</td></tr>
<tr><td>testBumpTransactionalEpochWithTV2Enabled(String, String, boolean)[1]</td><td>85.05%</td><td>14.95%</td><td>582</td><td>passed</td></tr>
<tr><td>testBumpTransactionalEpochWithTV2Disabled(String, String, boolean)[1]</td><td>91.07%</td><td>8.93%</td><td>582</td><td>passed</td></tr>
<tr><td>testBumpTransactionalEpochWithTV2Disabled(String, String, boolean)[2]</td><td>95.02%</td><td>4.98%</td><td>582</td><td>passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">kafka.api.TransactionsTest</td></tr>
<tr><td></td><td>7</td><td>27.07%</td><td>28.36%</td><td>580</td></tr>
<tr><td colspan="5">Build Outcomes: Passed: 423 | Failed: 9 | Flaky: 148</td></tr>
<tr><td colspan="5">Test Methods (Last 7 Days):</td></tr>
<tr><td>Method</td><td>Success Rate</td><td>Failure Rate</td><td>Total Runs</td><td>Recent Status</td></tr>
<tr><td>testBumpTransactionalEpochWithTV2Enabled(String, String, boolean)[1]</td><td>87.59%</td><td>12.41%</td><td>580</td><td>passed</td></tr>
<tr><td>testBumpTransactionalEpochWithTV2Enabled(String, String, boolean)[2]</td><td>89.66%</td><td>10.34%</td><td>580</td><td>passed</td></tr>
<tr><td>testBumpTransactionalEpochWithTV2Disabled(String, String, boolean)[1]</td><td>93.45%</td><td>6.55%</td><td>580</td><td>passed</td></tr>
<tr><td>testBumpTransactionalEpochWithTV2Disabled(String, String, boolean)[2]</td><td>95.00%</td><td>5.00%</td><td>580</td><td>passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">org.apache.kafka.common.network.Tls13SelectorTest</td></tr>
<tr><td></td><td>7</td><td>25.00%</td><td>25.00%</td><td>4</td></tr>
<tr><td colspan="5">Build Outcomes: Passed: 3 | Failed: 0 | Flaky: 1</td></tr>
<tr><td colspan="5">Test Methods (Last 7 Days):</td></tr>
<tr><td>Method</td><td>Success Rate</td><td>Failure Rate</td><td>Total Runs</td><td>Recent Status</td></tr>
<tr><td>testCloseOldestConnection()</td><td>75.00%</td><td>25.00%</td><td>4</td><td>passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">org.apache.kafka.common.network.SslTransportLayerTest</td></tr>
<tr><td></td><td>7</td><td>25.00%</td><td>25.00%</td><td>4</td></tr>
<tr><td colspan="5">Build Outcomes: Passed: 3 | Failed: 0 | Flaky: 1</td></tr>
<tr><td colspan="5">Test Methods (Last 7 Days):</td></tr>
<tr><td>Method</td><td>Success Rate</td><td>Failure Rate</td><td>Total Runs</td><td>Recent Status</td></tr>
<tr><td>testSelectorPollReadSize(Args)[1]</td><td>75.00%</td><td>25.00%</td><td>4</td><td>passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">org.apache.kafka.clients.consumer.internals.AbstractCoordinatorTest</td></tr>
<tr><td></td><td>7</td><td>16.67%</td><td>25.00%</td><td>6</td></tr>
<tr><td colspan="5">Build Outcomes: Passed: 5 | Failed: 0 | Flaky: 1</td></tr>
<tr><td colspan="5">Test Methods (Last 7 Days):</td></tr>
<tr><td>Method</td><td>Success Rate</td><td>Failure Rate</td><td>Total Runs</td><td>Recent Status</td></tr>
<tr><td>testWakeupAfterSyncGroupReceived()</td><td>75.00%</td><td>25.00%</td><td>4</td><td>passed</td></tr>
<tr><td colspan="5"> </td></tr>
<tr><td colspan="5">kafka.api.PlaintextConsumerTest</td></tr>
<tr><td></td><td>7</td><td>16.49%</td><td>17.10%</td><td>582</td></tr>
<tr><td colspan="5">Build Outcomes: Passed: 486 | Failed: 0 | Flaky: 96</td></tr>
<tr><td colspan="5">Test Methods (Last 7 Days):</td></tr>
<tr><td>Method</td><td>Success Rate</td><td>Failure Rate</td><td>Total Runs</td><td>Recent Status</td></tr>
<tr><td>testCloseLeavesGroupOnInterrupt(String, String)[2]</td><td>89.18%</td><td>10.82%</td><td>582</td><td>passed</td></tr>
<tr><td>testCoordinatorFailover(String, String)[2]</td><td>97.08%</td><td>2.92%</td><td>582</td><td>passed</td></tr>
<tr><td>testCloseLeavesGroupOnInterrupt(String, String)[1]</td><td>97.42%</td><td>2.58%</td><td>582</td><td>passed</td></tr>
<tr><td>testCoordinatorFailover(String, String)[1]</td><td>97.59%</td><td>2.41%</td><td>582</td><td>passed</td></tr>
<tr><td colspan="5"> </td></tr>
</table>
<details>
<summary>Detailed Execution History</summary>
### org.apache.kafka.tiered.storage.integration.OffloadAndTxnConsumeFromLeaderTest
#### executeTieredStorageTest(String, String)[1]
Recent Executions:
```
Date/Time (UTC) Outcome Build ID
--------------------------------------------
2024-12-15 08:00 failed lkhdsxpqn3mpa
2024-12-15 08:01 flaky sqgdbmoepagkq
```
#### executeTieredStorageTest(String, String)[2]
Recent Executions:
```
Date/Time (UTC) Outcome Build ID
--------------------------------------------
2024-12-15 08:00 flaky lkhdsxpqn3mpa
2024-12-15 08:01 flaky sqgdbmoepagkq
```
### org.apache.kafka.tiered.storage.integration.TransactionsWithTieredStoreTest
#### testBumpTransactionalEpochWithTV2Disabled(String, String, boolean)[2]
Recent Executions:
```
Date/Time (UTC) Outcome Build ID
--------------------------------------------
2024-12-16 06:48 passed t7lom5cz5smzk
2024-12-16 11:28 passed aoj3mrcyknile
2024-12-16 11:29 flaky 2is7idmt3mbmy
2024-12-16 14:48 passed 5abs5l5dr37yq
2024-12-16 14:49 passed irnmfsqtis6by
```
#### testReadCommittedConsumerShouldNotSeeUndecidedData(String, String)[2]
Recent Executions:
```
Date/Time (UTC) Outcome Build ID
--------------------------------------------
2024-12-16 06:48 passed t7lom5cz5smzk
2024-12-16 11:28 passed aoj3mrcyknile
2024-12-16 11:29 passed 2is7idmt3mbmy
2024-12-16 14:48 passed 5abs5l5dr37yq
2024-12-16 14:49 flaky irnmfsqtis6by
```
#### testBumpTransactionalEpochWithTV2Disabled(String, String, boolean)[1]
Recent Executions:
```
Date/Time (UTC) Outcome Build ID
--------------------------------------------
2024-12-16 06:48 passed t7lom5cz5smzk
2024-12-16 11:28 passed aoj3mrcyknile
2024-12-16 11:29 passed 2is7idmt3mbmy
2024-12-16 14:48 passed 5abs5l5dr37yq
2024-12-16 14:49 passed irnmfsqtis6by
```
#### testBumpTransactionalEpochWithTV2Enabled(String, String, boolean)[1]
Recent Executions:
```
Date/Time (UTC) Outcome Build ID
--------------------------------------------
2024-12-16 06:48 passed t7lom5cz5smzk
2024-12-16 11:28 passed aoj3mrcyknile
2024-12-16 11:29 flaky 2is7idmt3mbmy
2024-12-16 14:48 passed 5abs5l5dr37yq
2024-12-16 14:49 passed irnmfsqtis6by
```
#### testBumpTransactionalEpochWithTV2Enabled(String, String, boolean)[2]
Recent Executions:
```
Date/Time (UTC) Outcome Build ID
--------------------------------------------
2024-12-16 06:48 passed t7lom5cz5smzk
2024-12-16 11:28 passed aoj3mrcyknile
2024-12-16 11:29 passed 2is7idmt3mbmy
2024-12-16 14:48 flaky 5abs5l5dr37yq
2024-12-16 14:49 passed irnmfsqtis6by
```
### kafka.api.TransactionsTest
#### testBumpTransactionalEpochWithTV2Disabled(String, String, boolean)[2]
Recent Executions:
```
Date/Time (UTC) Outcome Build ID
--------------------------------------------
2024-12-16 06:48 passed t7lom5cz5smzk
2024-12-16 11:28 passed aoj3mrcyknile
2024-12-16 11:29 passed 2is7idmt3mbmy
2024-12-16 14:48 passed 5abs5l5dr37yq
2024-12-16 14:49 passed irnmfsqtis6by
```
#### testBumpTransactionalEpochWithTV2Disabled(String, String, boolean)[1]
Recent Executions:
```
Date/Time (UTC) Outcome Build ID
--------------------------------------------
2024-12-16 06:48 passed t7lom5cz5smzk
2024-12-16 11:28 passed aoj3mrcyknile
2024-12-16 11:29 passed 2is7idmt3mbmy
2024-12-16 14:48 passed 5abs5l5dr37yq
2024-12-16 14:49 passed irnmfsqtis6by
```
#### testBumpTransactionalEpochWithTV2Enabled(String, String, boolean)[1]
Recent Executions:
```
Date/Time (UTC) Outcome Build ID
--------------------------------------------
2024-12-16 06:48 passed t7lom5cz5smzk
2024-12-16 11:28 passed aoj3mrcyknile
2024-12-16 11:29 passed 2is7idmt3mbmy
2024-12-16 14:48 passed 5abs5l5dr37yq
2024-12-16 14:49 passed irnmfsqtis6by
```
#### testBumpTransactionalEpochWithTV2Enabled(String, String, boolean)[2]
Recent Executions:
```
Date/Time (UTC) Outcome Build ID
--------------------------------------------
2024-12-16 06:48 passed t7lom5cz5smzk
2024-12-16 11:28 passed aoj3mrcyknile
2024-12-16 11:29 passed 2is7idmt3mbmy
2024-12-16 14:48 passed 5abs5l5dr37yq
2024-12-16 14:49 passed irnmfsqtis6by
```
### org.apache.kafka.common.network.Tls13SelectorTest
#### testCloseOldestConnection()
Recent Executions:
```
Date/Time (UTC) Outcome Build ID
--------------------------------------------
2024-12-12 20:45 flaky hhdqo5lixoioi
2024-12-12 20:45 passed 4gllln4top7n6
2024-12-12 22:22 passed ct6a5b7wnnosw
2024-12-12 22:22 passed igseo3kh3m2z6
```
### org.apache.kafka.common.network.SslTransportLayerTest
#### testSelectorPollReadSize(Args)[1]
Recent Executions:
```
Date/Time (UTC) Outcome Build ID
--------------------------------------------
2024-12-12 20:45 passed hhdqo5lixoioi
2024-12-12 20:45 passed 4gllln4top7n6
2024-12-12 22:22 flaky ct6a5b7wnnosw
2024-12-12 22:22 passed igseo3kh3m2z6
```
### org.apache.kafka.clients.consumer.internals.AbstractCoordinatorTest
#### testWakeupAfterSyncGroupReceived()
Recent Executions:
```
Date/Time (UTC) Outcome Build ID
--------------------------------------------
2024-12-12 20:45 passed hhdqo5lixoioi
2024-12-12 20:45 flaky 4gllln4top7n6
2024-12-12 22:22 passed ct6a5b7wnnosw
2024-12-12 22:22 passed igseo3kh3m2z6
```
### kafka.api.PlaintextConsumerTest
#### testCoordinatorFailover(String, String)[2]
Recent Executions:
```
Date/Time (UTC) Outcome Build ID
--------------------------------------------
2024-12-16 06:48 passed t7lom5cz5smzk
2024-12-16 11:28 passed aoj3mrcyknile
2024-12-16 11:29 passed 2is7idmt3mbmy
2024-12-16 14:48 passed 5abs5l5dr37yq
2024-12-16 14:49 passed irnmfsqtis6by
```
#### testCoordinatorFailover(String, String)[1]
Recent Executions:
```
Date/Time (UTC) Outcome Build ID
--------------------------------------------
2024-12-16 06:48 passed t7lom5cz5smzk
2024-12-16 11:28 passed aoj3mrcyknile
2024-12-16 11:29 passed 2is7idmt3mbmy
2024-12-16 14:48 passed 5abs5l5dr37yq
2024-12-16 14:49 passed irnmfsqtis6by
```
#### testCloseLeavesGroupOnInterrupt(String, String)[1]
Recent Executions:
```
Date/Time (UTC) Outcome Build ID
--------------------------------------------
2024-12-16 06:48 passed t7lom5cz5smzk
2024-12-16 11:28 passed aoj3mrcyknile
2024-12-16 11:29 passed 2is7idmt3mbmy
2024-12-16 14:48 passed 5abs5l5dr37yq
2024-12-16 14:49 passed irnmfsqtis6by
```
#### testCloseLeavesGroupOnInterrupt(String, String)[2]
Recent Executions:
```
Date/Time (UTC) Outcome Build ID
--------------------------------------------
2024-12-16 06:48 passed t7lom5cz5smzk
2024-12-16 11:28 passed aoj3mrcyknile
2024-12-16 11:29 passed 2is7idmt3mbmy
2024-12-16 14:48 passed 5abs5l5dr37yq
2024-12-16 14:49 passed irnmfsqtis6by
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] flaky test report improvements [kafka]
Posted by "santhoshct (via GitHub)" <gi...@apache.org>.
santhoshct commented on PR #18212:
URL: https://github.com/apache/kafka/pull/18212#issuecomment-2547713388
I was also thinking about removing the high priority quarantine test report. the layout looks uniform. I have added it and updated the results in PR description.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] flaky test report improvements [kafka]
Posted by "santhoshct (via GitHub)" <gi...@apache.org>.
santhoshct commented on PR #18212:
URL: https://github.com/apache/kafka/pull/18212#issuecomment-2551882843
Updated with the changes - https://gist.github.com/santhoshct/f801b31c09adc02242c003889ddecf21
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] flaky test report improvements [kafka]
Posted by "mumrah (via GitHub)" <gi...@apache.org>.
mumrah commented on PR #18212:
URL: https://github.com/apache/kafka/pull/18212#issuecomment-2551370453
Thanks @santhoshct! Last request -- can you put the develocity link in a separate column so it aligns? It could be the rightmost column.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] flaky test report improvements [kafka]
Posted by "santhoshct (via GitHub)" <gi...@apache.org>.
santhoshct commented on PR #18212:
URL: https://github.com/apache/kafka/pull/18212#issuecomment-2550801322
sample report https://gist.github.com/santhoshct/169fcf56ed17e106e60316d9a708da1f
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] flaky test report improvements [kafka]
Posted by "mumrah (via GitHub)" <gi...@apache.org>.
mumrah commented on PR #18212:
URL: https://github.com/apache/kafka/pull/18212#issuecomment-2546495177
This looks _excellent_! The links will be super helpful.
Looks like the "High-Priority Quarantined Tests" table is mostly redundant with the summary table at the top. Actually, with the collapsed details sections, I think we can restructure things a little bit.
How about the following layout:
```
Most Problematic Tests
<1 sentence text description>
<table>
<summary><details>
---
Flaky Regressions
<1 sentence text description>
<table>
<summary><details>
---
Cleared Tests
<1 sentence text description>
<table>
<summary><details>
```
I think this will give the report a fairly uniform feel and make it a bit more intuitive to navigate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] flaky test report improvements [kafka]
Posted by "santhoshct (via GitHub)" <gi...@apache.org>.
santhoshct commented on PR #18212:
URL: https://github.com/apache/kafka/pull/18212#issuecomment-2550800936
It was my mistake. I forgot to push the code and just updated the test results in PR description. anyways I have pushed the code now and also modified the hyperlinks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] flaky test report improvements [kafka]
Posted by "mumrah (via GitHub)" <gi...@apache.org>.
mumrah merged PR #18212:
URL: https://github.com/apache/kafka/pull/18212
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org