[jira] [Commented] (DRILL-1536) Selecting a null valued field fails on json input format with an irrelevant error message

["Jason Altekruse (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-1536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14171635#comment-14171635 ] 

Jason Altekruse commented on DRILL-1536:
----------------------------------------

While this is a bug, the problem is not with how we are handling nulls in general, but instead in the case where the selected columns have nothing but nulls in them. The default behavior when encountering a null is to only check that the value can be represented in the allocated vector. However in the case where we have not yet hit a non-null value, we do not know a type and have no vector to check (there is no representation of a typeless null in Drill today). We need to have special handling added for the case where only nulls are found, because it is currently resulting in no vectors being added to the schema.

> Selecting a null valued field fails on json input format with an irrelevant error message
> -----------------------------------------------------------------------------------------
>
>                 Key: DRILL-1536
>                 URL: https://issues.apache.org/jira/browse/DRILL-1536
>             Project: Apache Drill
>          Issue Type: Bug
>            Reporter: Hanifi Gunes
>
> Running 
> {code:sql}
> select t.id from dfs.`data/simple.json` t;
> {code}
> against
> {panel:title=simple.json}
> \{"id":null\}
> {panel}
> throws
> {panel}
> org.apache.drill.common.exceptions.DrillRuntimeException: Record is too big to fit into allocated ValueVector
> 	at org.apache.drill.exec.store.easy.json.JSONRecordReader2.next(JSONRecordReader2.java:121)
> 	at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:158)
> 	at org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:117)
> 	at org.apache.drill.exec.physical.impl.producer.ProducerConsumerBatch$Producer.run(ProducerConsumerBatch.java:128)
> 	at java.lang.Thread.run(Thread.java:745)
> {panel}
> We need to make sure a value vector for the field `id` is allocated before attempting to manipulate the vector. Currently JsonReader does nothing beyond a bound check when encountered with a null value. We should implement proper null value handling.
> Also the above error message should precisely reflect the root cause of the problem. Thrown exception in this case is irrelevant and misleading.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)