[jira] [Created] (DRILL-7685) Case statement marking column as required in parquet metadata

["Nitin Pawar (Jira)" <ji...@apache.org>]

Nitin Pawar created DRILL-7685:
----------------------------------

             Summary: Case statement marking column as required in parquet metadata
                 Key: DRILL-7685
                 URL: https://issues.apache.org/jira/browse/DRILL-7685
             Project: Apache Drill
          Issue Type: Improvement
          Components: Storage - Parquet
    Affects Versions: 1.16.0
            Reporter: Nitin Pawar


We use apache drill for multi step processing.

In one of the steps we have query as below

~create table dfs.tmp.`/t2` as select employee_id, case when department_id is not null then 1 else 2 end as case_output from cp.`employee.json`;~

This provides output as 

employee_id: OPTIONAL INT64 R:0 D:1
case_output: REQUIRED INT32 R:0 D:0

If we remove the end statement from case it does mark the column as optional.

 

We feed this output to covariance function and because of this we get an error like below 

Error: Missing function implementation: [covariance(BIGINT-OPTIONAL, INT-REQUIRED)]. Full expression: --UNKNOWN EXPRESSION--

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)