[jira] [Commented] (DRILL-5283) Support "is not present" as subtype of "is null" for JSON data

["Julian Hyde (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15876356#comment-15876356 ] 

Julian Hyde commented on DRILL-5283:
------------------------------------

Consider calling this "is empty". MDX and HBase/Phoenix have concepts of empty cells, and we should share concepts wherever possible. Otherwise you no longer have a data model; you are bubbling up details of the data format.

> Support "is not present" as subtype of "is null" for JSON data
> --------------------------------------------------------------
>
>                 Key: DRILL-5283
>                 URL: https://issues.apache.org/jira/browse/DRILL-5283
>             Project: Apache Drill
>          Issue Type: Improvement
>    Affects Versions: 1.10.0
>            Reporter: Paul Rogers
>
> JSON files consist of a series of "objects", each of which has name/value pairs. Values can be in one of three states:
> * Not present (the value does not appear)
> * Null (the name appears and the value is null)
> * Non-null (the field is one of the JSON data types)
> Drill, however, has only a single null state and so Drill collapses "not present" and "null" into the same state.
> The not-present and present-but-null states work identically for calculations inside Drill. But, when doing a CTAS from JSON to JSON, the collapsed state means that the user does not get out of Drill what was put in: all null values either appear as null values, or do not appear at all (depending on Drill version.)
> This ticket asks to repurpose the "bit" fields in nullable vectors. Rename the vector to "nullState". Then, use these values:
> * 0: value is set
> * 1: value is null
> * 3: value is not present
> The column is null if the null state is non-zero. The column is not null if the null state is 0.
> This change requires reversing the "polarity" of the bit field, and so is a major change.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)