[jira] [Reopened] (DRILL-3810) Filesystem plugin's support for file format's schema

["Chun Chang (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/DRILL-3810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chun Chang reopened DRILL-3810:
-------------------------------

Can someone explain to me what is the expected behavior for this fix? Against schema based file, we will validate query against file schema and fail the query before executing it if the validation fail, is that right? If I query a parquet file for a column that does not exist, the query should fail fast? It's not happening. Query a column that does not exist returns null.

{noformat}
0: jdbc:drill:schema=dfs.drillTestDir> select version, commit_id from sys.version;
+-----------------+-------------------------------------------+
|     version     |                 commit_id                 |
+-----------------+-------------------------------------------+
| 1.4.0-SNAPSHOT  | b9068117177c3b47025f52c00f67938e0c3e4732  |
+-----------------+-------------------------------------------+

0: jdbc:drill:schema=dfs.drillTestDir> select c_row, c_int from dfs.`/drill/testdata/numerical/data` limit 1;
+--------+--------+
| c_row  | c_int  |
+--------+--------+
| 1      | 0      |
+--------+--------+
1 row selected (0.355 seconds)
0: jdbc:drill:schema=dfs.drillTestDir> select c_row, c_column_does_not_exist from dfs.`/drill/testdata/numerical/data` limit 1;
+--------+--------------------------+
| c_row  | c_column_does_not_exist  |
+--------+--------------------------+
| 1      | null                     |
+--------+--------------------------+
{noformat}

Please close the JIRA if my understanding was incorrect.

> Filesystem plugin's support for file format's schema
> ----------------------------------------------------
>
>                 Key: DRILL-3810
>                 URL: https://issues.apache.org/jira/browse/DRILL-3810
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Storage - JSON, Storage - Other, Storage - Parquet, Storage - Text & CSV
>            Reporter: Bhallamudi Venkata Siva Kamesh
>            Assignee: Bhallamudi Venkata Siva Kamesh
>             Fix For: 1.3.0
>
>
> Filesystem Plugin supports multiple type of file formats like
> 					* json
> 					* avro
> 					* text (csv|psv|tsv)
> 					* parquet
> and can support any type of file formats.
> Among these file formats, some of the file formats are schema based like
> *avro* and *parquet* and some of them are schema less like *json*.
> For schema based file formats, Drill should have capability to validate the query against file schema, before start executing the query.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)