You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2021/06/18 20:50:39 UTC
[GitHub] [kafka] hachikuji commented on a change in pull request #10836: KAFKA-12906 - Added RecordDeserializationException containing partition and offset
hachikuji commented on a change in pull request #10836:
URL: https://github.com/apache/kafka/pull/10836#discussion_r653786412
##########
File path: clients/src/test/java/org/apache/kafka/clients/consumer/KafkaConsumerTest.java
##########
@@ -186,6 +188,101 @@ public void testMetricsReporterAutoGeneratedClientId() {
consumer.close();
}
+ @Test
+ public void testPollReturnsRecords() {
+ KafkaConsumer<String, String> consumer = setUpConsumerWithRecordsToPoll(tp0, 5);
+
+ ConsumerRecords<String, String> records = consumer.poll(Duration.ZERO);
+
+ assertEquals(records.count(), 5);
+ assertEquals(records.partitions(), new HashSet<>(Collections.singletonList(tp0)));
+ assertEquals(records.records(tp0).size(), 5);
+
+ consumer.close(Duration.ofMillis(0));
+ }
+
+ @Test
+ public void testPollReturnsRecordsUpUntilError() {
+ int invalidRecordNumber = 4;
+ StringDeserializer deserializer = mockErrorDeserializer(invalidRecordNumber);
+
+ KafkaConsumer<String, String> consumer = setUpConsumerWithRecordsToPoll(tp0, 5, deserializer);
+
+ ConsumerRecords<String, String> records = consumer.poll(Duration.ZERO);
+
+ assertEquals(invalidRecordNumber - 1, records.count());
+ assertEquals(new HashSet<>(Collections.singletonList(tp0)), records.partitions());
+ assertEquals(invalidRecordNumber - 1, records.records(tp0).size());
+ long lastValidOffset = records.records(tp0).get(records.records(tp0).size() - 1).offset();
+ assertEquals(invalidRecordNumber - 2, lastValidOffset);
+
+ consumer.close(Duration.ofMillis(0));
+ }
+
+ @Test
+ public void testSecondPollWithDeserializationErrorThrowsRecordDeserializationException() {
+ int invalidRecordNumber = 4;
+ int invalidRecordOffset = 3;
+ StringDeserializer deserializer = mockErrorDeserializer(invalidRecordNumber);
+
+ KafkaConsumer<String, String> consumer = setUpConsumerWithRecordsToPoll(tp0, 5, deserializer);
+ ConsumerRecords<String, String> records = consumer.poll(Duration.ZERO);
+
+ assertEquals(invalidRecordNumber - 1, records.count());
+ assertEquals(new HashSet<>(Collections.singletonList(tp0)), records.partitions());
Review comment:
Worked fine when I tried it locally:
```java
assertEquals(Collections.singleton(tp0), records.partitions());
```
##########
File path: clients/src/test/java/org/apache/kafka/clients/consumer/KafkaConsumerTest.java
##########
@@ -186,6 +188,101 @@ public void testMetricsReporterAutoGeneratedClientId() {
consumer.close();
}
+ @Test
+ public void testPollReturnsRecords() {
+ KafkaConsumer<String, String> consumer = setUpConsumerWithRecordsToPoll(tp0, 5);
+
+ ConsumerRecords<String, String> records = consumer.poll(Duration.ZERO);
+
+ assertEquals(records.count(), 5);
+ assertEquals(records.partitions(), new HashSet<>(Collections.singletonList(tp0)));
+ assertEquals(records.records(tp0).size(), 5);
+
+ consumer.close(Duration.ofMillis(0));
+ }
+
+ @Test
+ public void testPollReturnsRecordsUpUntilError() {
+ int invalidRecordNumber = 4;
+ StringDeserializer deserializer = mockErrorDeserializer(invalidRecordNumber);
+
+ KafkaConsumer<String, String> consumer = setUpConsumerWithRecordsToPoll(tp0, 5, deserializer);
+
+ ConsumerRecords<String, String> records = consumer.poll(Duration.ZERO);
+
+ assertEquals(invalidRecordNumber - 1, records.count());
+ assertEquals(new HashSet<>(Collections.singletonList(tp0)), records.partitions());
+ assertEquals(invalidRecordNumber - 1, records.records(tp0).size());
+ long lastValidOffset = records.records(tp0).get(records.records(tp0).size() - 1).offset();
+ assertEquals(invalidRecordNumber - 2, lastValidOffset);
+
+ consumer.close(Duration.ofMillis(0));
+ }
+
+ @Test
+ public void testSecondPollWithDeserializationErrorThrowsRecordDeserializationException() {
+ int invalidRecordNumber = 4;
+ int invalidRecordOffset = 3;
+ StringDeserializer deserializer = mockErrorDeserializer(invalidRecordNumber);
+
+ KafkaConsumer<String, String> consumer = setUpConsumerWithRecordsToPoll(tp0, 5, deserializer);
+ ConsumerRecords<String, String> records = consumer.poll(Duration.ZERO);
+
+ assertEquals(invalidRecordNumber - 1, records.count());
+ assertEquals(new HashSet<>(Collections.singletonList(tp0)), records.partitions());
Review comment:
You could submit a minor follow-up if you like.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org