You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "artemlivshits (via GitHub)" <gi...@apache.org> on 2024/02/05 23:03:41 UTC
[PR] MINOR: extend transaction unit test to validate drain [kafka]
artemlivshits opened a new pull request, #15320:
URL: https://github.com/apache/kafka/pull/15320
There is a test already that checks that transactional messages are not drained when partition is not added, this change just logically completes the test to also show that messages can be drained after parition is added.
*More detailed description of your change,
if necessary. The PR title and PR message become
the squashed commit message, so use a separate
comment to ping reviewers.*
*Summary of testing strategy (including rationale)
for the feature or bug fix. Unit and/or integration
tests are expected for any behaviour change and
system tests should be considered for larger changes.*
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: extend transaction unit test to validate drain [kafka]
Posted by "jolshan (via GitHub)" <gi...@apache.org>.
jolshan commented on code in PR #15320:
URL: https://github.com/apache/kafka/pull/15320#discussion_r1497943369
##########
clients/src/test/java/org/apache/kafka/clients/producer/internals/TransactionManagerTest.java:
##########
@@ -2548,6 +2548,23 @@ public void testRaiseErrorWhenNoPartitionsPendingOnDrain() throws InterruptedExc
// We shouldn't drain batches which haven't been added to the transaction yet.
assertTrue(drainedBatches.containsKey(node1.id()));
assertTrue(drainedBatches.get(node1.id()).isEmpty());
+
+ // Let's now add the partition, flush and try to drain again.
+ transactionManager.maybeAddPartition(tp0);
+ accumulator.beginFlush();
+
+ drainedBatches = accumulator.drain(metadataMock, nodes, Integer.MAX_VALUE,
Review Comment:
thanks folks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: extend transaction unit test to validate drain [kafka]
Posted by "cadonna (via GitHub)" <gi...@apache.org>.
cadonna commented on code in PR #15320:
URL: https://github.com/apache/kafka/pull/15320#discussion_r1497207704
##########
clients/src/test/java/org/apache/kafka/clients/producer/internals/TransactionManagerTest.java:
##########
@@ -2548,6 +2548,23 @@ public void testRaiseErrorWhenNoPartitionsPendingOnDrain() throws InterruptedExc
// We shouldn't drain batches which haven't been added to the transaction yet.
assertTrue(drainedBatches.containsKey(node1.id()));
assertTrue(drainedBatches.get(node1.id()).isEmpty());
+
+ // Let's now add the partition, flush and try to drain again.
+ transactionManager.maybeAddPartition(tp0);
+ accumulator.beginFlush();
+
+ drainedBatches = accumulator.drain(metadataMock, nodes, Integer.MAX_VALUE,
Review Comment:
This PR should fix the compilation error: https://github.com/apache/kafka/pull/15405
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: extend transaction unit test to validate drain [kafka]
Posted by "jolshan (via GitHub)" <gi...@apache.org>.
jolshan merged PR #15320:
URL: https://github.com/apache/kafka/pull/15320
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: extend transaction unit test to validate drain [kafka]
Posted by "sjhajharia (via GitHub)" <gi...@apache.org>.
sjhajharia commented on code in PR #15320:
URL: https://github.com/apache/kafka/pull/15320#discussion_r1497137732
##########
clients/src/test/java/org/apache/kafka/clients/producer/internals/TransactionManagerTest.java:
##########
@@ -2548,6 +2548,23 @@ public void testRaiseErrorWhenNoPartitionsPendingOnDrain() throws InterruptedExc
// We shouldn't drain batches which haven't been added to the transaction yet.
assertTrue(drainedBatches.containsKey(node1.id()));
assertTrue(drainedBatches.get(node1.id()).isEmpty());
+
+ // Let's now add the partition, flush and try to drain again.
+ transactionManager.maybeAddPartition(tp0);
+ accumulator.beginFlush();
+
+ drainedBatches = accumulator.drain(metadataMock, nodes, Integer.MAX_VALUE,
Review Comment:
@artemlivshits There seems to be an error in this line. The variable `metadataMock` is not defined and that causes the test to break. I think the variable you were looking for is `metadataCache`. Could you please check the same?
Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: extend transaction unit test to validate drain [kafka]
Posted by "cadonna (via GitHub)" <gi...@apache.org>.
cadonna commented on code in PR #15320:
URL: https://github.com/apache/kafka/pull/15320#discussion_r1497201056
##########
clients/src/test/java/org/apache/kafka/clients/producer/internals/TransactionManagerTest.java:
##########
@@ -2548,6 +2548,23 @@ public void testRaiseErrorWhenNoPartitionsPendingOnDrain() throws InterruptedExc
// We shouldn't drain batches which haven't been added to the transaction yet.
assertTrue(drainedBatches.containsKey(node1.id()));
assertTrue(drainedBatches.get(node1.id()).isEmpty());
+
+ // Let's now add the partition, flush and try to drain again.
+ transactionManager.maybeAddPartition(tp0);
+ accumulator.beginFlush();
+
+ drainedBatches = accumulator.drain(metadataMock, nodes, Integer.MAX_VALUE,
Review Comment:
@sjhajharia I also experienced this issue. Variable `metadataMock` was probably removed after the CI build was run and the PR was merged. The CI build for this PR does not have a compilation error.
The PR that removed the variable is https://github.com/apache/kafka/commit/ff90f78c700c582f9800013faad827c36b45ceb7#diff-36184945d0619b12edaf88d58f6f7668d543cbc534948d381017eb978a033a59
I will try to fix this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: extend transaction unit test to validate drain [kafka]
Posted by "cadonna (via GitHub)" <gi...@apache.org>.
cadonna commented on code in PR #15320:
URL: https://github.com/apache/kafka/pull/15320#discussion_r1497317344
##########
clients/src/test/java/org/apache/kafka/clients/producer/internals/TransactionManagerTest.java:
##########
@@ -2548,6 +2548,23 @@ public void testRaiseErrorWhenNoPartitionsPendingOnDrain() throws InterruptedExc
// We shouldn't drain batches which haven't been added to the transaction yet.
assertTrue(drainedBatches.containsKey(node1.id()));
assertTrue(drainedBatches.get(node1.id()).isEmpty());
+
+ // Let's now add the partition, flush and try to drain again.
+ transactionManager.maybeAddPartition(tp0);
+ accumulator.beginFlush();
+
+ drainedBatches = accumulator.drain(metadataMock, nodes, Integer.MAX_VALUE,
Review Comment:
Merged
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org