You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "JiayaoS (via GitHub)" <gi...@apache.org> on 2026/05/22 10:23:46 UTC
[PR] MINOR: Reduce close delay in testFailureToFenceEpoch [kafka]
JiayaoS opened a new pull request, #22347:
URL: https://github.com/apache/kafka/pull/22347
Reduce the request timeout for the producer used by testFailureToFenceEpoch so graceful close does not wait on the default 30s timeout after the fencing probe leaves the producer in an empty transaction.
Tested by running testFailureToFenceEpochWithTV1 and testFailureToFenceEpochWithTV2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Reduce close delay in testFailureToFenceEpoch [kafka]
Posted by "JiayaoS (via GitHub)" <gi...@apache.org>.
JiayaoS commented on code in PR #22347:
URL: https://github.com/apache/kafka/pull/22347#discussion_r3291393051
##########
storage/src/test/java/org/apache/kafka/tiered/storage/integration/TransactionsTestHelper.java:
##########
@@ -923,7 +923,9 @@ public static void testFailureToFenceEpoch(
long producerId;
int initialProducerEpoch;
- try (var producer1 = createTransactionalProducer(clusterInstance, "transactional-producer")) {
Review Comment:
> How about producer1.close(duration 5 secs) in finally ? This will force a hard close after the time out, instead of waiting for pending reqs.
Thanks, that’s a fair point. I considered `close(Duration.ofSeconds(5))` as well, but it changes the cleanup behavior to force-close after the timeout. The current approach keeps the try-with-resources close graceful, while bounding the pending requests with a shorter request timeout. Maybe we should decide which behavior is more appropriate for this test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Reduce close delay in testFailureToFenceEpoch [kafka]
Posted by "JiayaoS (via GitHub)" <gi...@apache.org>.
JiayaoS commented on PR #22347:
URL: https://github.com/apache/kafka/pull/22347#issuecomment-4517866422
I used the following loop to re-run both affected tests 10 times and summarize their runtimes. It removes only the test result directories between runs, so Gradle can keep compilation tasks up-to-date while still re-running the tests.
```bash
OUT=/tmp/fence-epoch-runs.tsv
XML=storage/build/test-results/test/TEST-org.apache.kafka.tiered.storage.integration.TransactionsTest.xml
: > "$OUT"
for run in $(seq 1 10); do
echo "===== run ${run}/10 ====="
rm -rf storage/build/test-results/test storage/build/reports/tests/test storage/build/test-results/binary/test
./gradlew :storage:test \
--tests org.apache.kafka.tiered.storage.integration.TransactionsTest.testFailureToFenceEpochWithTV1 \
--tests org.apache.kafka.tiered.storage.integration.TransactionsTest.testFailureToFenceEpochWithTV2
sed -n 's/.*<testcase name="\([^"]*testFailureToFenceEpochWithTV[12][^"]*\)".* time="\([^"]*\)".*/\1\t\2/p' "$XML" |
awk -F "\t" -v run="$run" '{
test = $1
sub(/^.*testFailureToFenceEpochWithTV/, "testFailureToFenceEpochWithTV", test)
sub(/ .*/, "", test)
print run "\t" test "\t" $2
}' >> "$OUT"
done
summarize() {
local label="$1"
awk -v label="$label" '
{ times[++count] = $1; sum += $1; if ($1 > max) max = $1 }
END {
if (count == 0) {
printf "%s: count=0\n", label
exit
}
if (count % 2 == 1) {
median = times[(count + 1) / 2]
} else {
median = (times[count / 2] + times[count / 2 + 1]) / 2
}
printf "%s: count=%d, max=%.3fs, median=%.3fs, avg=%.3fs\n", label, count, max, median, sum / count
}
'
}
for test_name in testFailureToFenceEpochWithTV1 testFailureToFenceEpochWithTV2; do
awk -v test="$test_name" '$2 == test { print $3 }' "$OUT" | sort -n | summarize "$test_name"
done
awk '{ print $3 }' "$OUT" | sort -n | summarize "ALL"
```
Results:
| Test | Count | Max | Median | Average |
| --- | ---: | ---: | ---: | ---: |
| `testFailureToFenceEpochWithTV1` | 10 | 19.959s | 18.305s | 18.570s |
| `testFailureToFenceEpochWithTV2` | 10 | 16.832s | 16.718s | 16.718s |
| All runs | 20 | 19.959s | 17.341s | 17.644s |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Reduce close delay in testFailureToFenceEpoch [kafka]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22347:
URL: https://github.com/apache/kafka/pull/22347#issuecomment-6487191458
This PR is being marked as stale since it has not had any activity in 90 days. If you
would like to keep this PR alive, please leave a comment asking for a review. If the PR has
merge conflicts, update it with the latest from the base branch.
<p>
If you are having difficulty finding a reviewer, please reach out on the
[mailing list](https://kafka.apache.org/contact).
<p>
If this PR is no longer valid or desired, please feel free to close it. If no activity
occurs in the next 30 days, it will be automatically closed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Reduce close delay in testFailureToFenceEpoch [kafka]
Posted by "chia7712 (via GitHub) via jira" <ji...@kafka.apache.org>.
chia7712 commented on PR #22347:
URL: https://github.com/apache/kafka/pull/22347#issuecomment-6801179319
It seems to me the root cause is the unnecessary waiting in the producer close, and I opened https://issues.apache.org/jira/browse/KAFKA-21471 to track it. We can revisit this patch once KAFKA-21471 is fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Reduce close delay in testFailureToFenceEpoch [kafka]
Posted by "muralibasani (via GitHub)" <gi...@apache.org>.
muralibasani commented on code in PR #22347:
URL: https://github.com/apache/kafka/pull/22347#discussion_r3288960453
##########
storage/src/test/java/org/apache/kafka/tiered/storage/integration/TransactionsTestHelper.java:
##########
@@ -923,7 +923,9 @@ public static void testFailureToFenceEpoch(
long producerId;
int initialProducerEpoch;
- try (var producer1 = createTransactionalProducer(clusterInstance, "transactional-producer")) {
Review Comment:
How about producer1.close(duration 5 secs) in finally ? This will force a hard close after the time out, instead of waiting for pending reqs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org