[jira] [Updated] (DRILL-6824) Drill Query on MapRDB JSON table failing on schema SchemaChangeException, the only distinct Values are NULL and Text

["Gautam Parai (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/DRILL-6824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gautam Parai updated DRILL-6824:
--------------------------------
    Description: 
Drill Query on MapR DB JSON Table or it View fails with a SchemaChangeException. The only distinct values is NULL and some Text.

The issue is that when Drill sees a NULL or does not see any values for a column in the first batch it assumes the type as NULLABLE INT. Subsequently, when the data shows up and it is different from NULLABLE INT there is a schema change. Operators e.g. aggregators etc. cannot handle such a Schema Change and throw a SchemaChangeException.

 

One of the short-term solution implemented in this fix: Add a CAST expression which will cast null values to the target type. Hence, we would never see a SchemaChange due to NULLs. However, the MapRDB Reader code was written differently than other reader and was hitting a SchemaChangeException. The code was changed to make it similar to other scans i.e. emit a new batch whenever a schema change is encountered.

  was:Drill Query on MapR DB JSON Table or it View fails with a SchemaChangeException. Th only distinct values is NULL and some Text.


> Drill Query on MapRDB JSON table failing on schema SchemaChangeException, the only distinct Values are NULL and Text
> --------------------------------------------------------------------------------------------------------------------
>
>                 Key: DRILL-6824
>                 URL: https://issues.apache.org/jira/browse/DRILL-6824
>             Project: Apache Drill
>          Issue Type: Bug
>    Affects Versions: 1.14.0, 1.15.0
>            Reporter: Gautam Parai
>            Assignee: Gautam Parai
>            Priority: Major
>             Fix For: 1.15.0
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> Drill Query on MapR DB JSON Table or it View fails with a SchemaChangeException. The only distinct values is NULL and some Text.
> The issue is that when Drill sees a NULL or does not see any values for a column in the first batch it assumes the type as NULLABLE INT. Subsequently, when the data shows up and it is different from NULLABLE INT there is a schema change. Operators e.g. aggregators etc. cannot handle such a Schema Change and throw a SchemaChangeException.
>  
> One of the short-term solution implemented in this fix: Add a CAST expression which will cast null values to the target type. Hence, we would never see a SchemaChange due to NULLs. However, the MapRDB Reader code was written differently than other reader and was hitting a SchemaChangeException. The code was changed to make it similar to other scans i.e. emit a new batch whenever a schema change is encountered.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)