You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@drill.apache.org by so...@apache.org on 2019/01/11 17:03:39 UTC
[drill] 01/04: DRILL-6918: Skip ensureAtLeastOneField when there
are no records
This is an automated email from the ASF dual-hosted git repository.
sorabh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/drill.git
commit 4b66226dc7e33d9173e6e714d5cf85beea1e4e46
Author: Abhishek Ravi <ab...@gmail.com>
AuthorDate: Fri Jan 4 00:04:32 2019 -0800
DRILL-6918: Skip ensureAtLeastOneField when there are no records
If none of the project / filter columns, exist in the vector, ensureAtLeastOneField (or the Scan operator) adds at least one field as nullable integer (or nullable varchar if `allTextmode` is enabled).
The downstream Filter operator would then go on to fail with `NumberFormatException` because it tries to convert empty fields to integers.
Since ensureAtLeastOneField is called after reading all the messages in a batch, it can be skipped if the batch is empty.
closes #1595
---
.../java/org/apache/drill/exec/store/kafka/KafkaRecordReader.java | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/contrib/storage-kafka/src/main/java/org/apache/drill/exec/store/kafka/KafkaRecordReader.java b/contrib/storage-kafka/src/main/java/org/apache/drill/exec/store/kafka/KafkaRecordReader.java
index 9559c3d..62e588c 100644
--- a/contrib/storage-kafka/src/main/java/org/apache/drill/exec/store/kafka/KafkaRecordReader.java
+++ b/contrib/storage-kafka/src/main/java/org/apache/drill/exec/store/kafka/KafkaRecordReader.java
@@ -119,7 +119,9 @@ public class KafkaRecordReader extends AbstractRecordReader {
}
}
- messageReader.ensureAtLeastOneField();
+ if (currentMessageCount > 0) {
+ messageReader.ensureAtLeastOneField();
+ }
writer.setValueCount(currentMessageCount);
logger.debug("Took {} ms to process {} records.", watch.elapsed(TimeUnit.MILLISECONDS), currentMessageCount);
logger.debug("Last offset consumed for {}:{} is {}", subScanSpec.getTopicName(), subScanSpec.getPartitionId(),