You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "rondagostino (via GitHub)" <gi...@apache.org> on 2023/05/16 15:13:16 UTC
[GitHub] [kafka] rondagostino opened a new pull request, #13724: MINOR: more TopicsImage tests
rondagostino opened a new pull request, #13724:
URL: https://github.com/apache/kafka/pull/13724
Adds additional testing for TopicsImage. For every image that we create, test that we can get to the final image with an intermediate stop at all intermediate images along the way. For example, if we have 3 records that comprise a final image, ensure that we can build that exact final image by:
* applying the first record against an empty image, creating an intermediate image, and then applying the final 2 records on top of that intermediate image
* applying the first two records against an empty image, creating an intermediate image, and then applying the final record on top of that intermediate image
So for any image created with N records, we test the N-1 ways to get to that final image.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] mumrah commented on a diff in pull request #13724: MINOR: more KRaft Metadata Image tests
Posted by "mumrah (via GitHub)" <gi...@apache.org>.
mumrah commented on code in PR #13724:
URL: https://github.com/apache/kafka/pull/13724#discussion_r1230101055
##########
metadata/src/test/java/org/apache/kafka/metadata/RecordTestUtils.java:
##########
@@ -95,6 +95,60 @@ public static void replayOne(
replayAll(target, Collections.singletonList(recordAndVersion));
}
+ public interface TestThroughAllIntermediateImagesLeadingToFinalImageHelper {
Review Comment:
Could we make this a concrete class? We could take in a supplier for an empty image and a function for creating a delta from an image. Also, I think we could add type parameters to avoid some of the casting. E.g.,
```java
class HelperThing<I, D> {
HelperThing(Supplier<I> emptyImageSupplier, Function<I, D> imageToDelta)
void test(I image, List<ApiMessageAndVersion> fromRecords);
}
```
then use it like
```java
HelperThing helper = new HelperThing(() -> TopicsImage.EMPTY, image -> new TopicsDelta(image));
helper.test(image, fromRecords);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] rondagostino commented on pull request #13724: MINOR: more KRaft Metadata Image tests
Posted by "rondagostino (via GitHub)" <gi...@apache.org>.
rondagostino commented on PR #13724:
URL: https://github.com/apache/kafka/pull/13724#issuecomment-1629024939
4 test failures, unrelated:
```
Build / JDK 8 and Scala 2.12 / testBalancePartitionLeaders() – org.apache.kafka.controller.QuorumControllerTest
13s
Build / JDK 17 and Scala 2.13 / testGracefulClose() – org.apache.kafka.clients.consumer.KafkaConsumerTest
2s
Build / JDK 20 and Scala 2.13 / testConnectorBoundary – org.apache.kafka.connect.integration.ExactlyOnceSourceIntegrationTest
1m 10s
Build / JDK 20 and Scala 2.13 / testBalancePartitionLeaders() – org.apache.kafka.controller.QuorumControllerTest
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] rondagostino commented on pull request #13724: MINOR: more KRaft Metadata Image tests
Posted by "rondagostino (via GitHub)" <gi...@apache.org>.
rondagostino commented on PR #13724:
URL: https://github.com/apache/kafka/pull/13724#issuecomment-1591687192
4 test failures on latest build are unrelated. This PR just changes tests, and all of the affected tests passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] rondagostino merged pull request #13724: MINOR: more KRaft Metadata Image tests
Posted by "rondagostino (via GitHub)" <gi...@apache.org>.
rondagostino merged PR #13724:
URL: https://github.com/apache/kafka/pull/13724
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] mumrah commented on pull request #13724: MINOR: more KRaft Metadata Image tests
Posted by "mumrah (via GitHub)" <gi...@apache.org>.
mumrah commented on PR #13724:
URL: https://github.com/apache/kafka/pull/13724#issuecomment-1584996747
@rondagostino I like the approach you've got here! It will definitely help us cover a lot of edge conditions in the replay and apply logic.
Do you think we could extend the idea to also vary the size of the batch being applied as we go along?
For example, with 6 records, you have 5 possibilities currently. If we add another dimension (m) to be the size of the batch being applied, you would get a lot more combinations. At n=3 (three initial records) instead of only {A, B, C} {D, E, F}, we would have:
* {A, B, C} {D} {E} {F} (m=1)
* {A, B, C} {D, E} {F} (m=2)
* {A, B, C} {D, E, F} (m=3)
that would give us something on the order of N^2 / 2 permutations.
WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] rondagostino commented on pull request #13724: MINOR: more KRaft Metadata Image tests
Posted by "rondagostino (via GitHub)" <gi...@apache.org>.
rondagostino commented on PR #13724:
URL: https://github.com/apache/kafka/pull/13724#issuecomment-1563388747
Lots of test failures, but all of the tests touched here succeeded, plus there are no changes to non-test code. So tests failures are unrelated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org