[jira] [Created] (DRILL-468) Support for FileSystem partitions

["Steven Phillips (JIRA)" <ji...@apache.org>]

Steven Phillips created DRILL-468:
-------------------------------------

             Summary: Support for FileSystem partitions
                 Key: DRILL-468
                 URL: https://issues.apache.org/jira/browse/DRILL-468
             Project: Apache Drill
          Issue Type: Bug
            Reporter: Steven Phillips


For filesystem partitioning, we want to use the existing directory structure of the data. So, if a selection is a directory that contains subdirectories, the name of the directory a given record was stored in can be included as a field in that record. For example, given this structure:

/data
  /a
    file.csv
  /b
    file.csv


select * from dfs.`/data`

will include a column named dir0, with possible values a and b. This can be extended to a hierarchy of partitions. For example,

/data
  /a
    /1
      file.csv
    /2
      file.csv
  /b
    file.csv


would have columns dir0 (with possible values a and b) and dir1 (with possible values 1, 2 and null).

The data type will always be VARCHAR for the partition columns.



--
This message was sent by Atlassian JIRA
(v6.2#6252)