[jira] [Commented] (DRILL-5414) Issue with Querying Directories

["Pabi Ambikainathan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-5414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16712049#comment-16712049 ] 

Pabi Ambikainathan commented on DRILL-5414:
-------------------------------------------

Also ran into this issue when trying to querying sub-directories in S3, using UNION as a workaround but it has limitations unfortunately.

> Issue with Querying Directories
> -------------------------------
>
>                 Key: DRILL-5414
>                 URL: https://issues.apache.org/jira/browse/DRILL-5414
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Functions - Drill
>    Affects Versions: 1.10.0
>         Environment: Kubernetes running Debian GNU/Linux 8 containers.
> openjdk version "1.8.0_111".
> AWS.
> Using s3 buckets
>            Reporter: Paul Makkar
>            Priority: Major
>
> Hi
> *Thanks for apache drill - it's pretty awesome :)
> I'm hoping to exploit drill directory querying and have structured my data archive in s3 to test this. However, I've got an issue using directory querying.
> My directory structure in s3 is like:
> s3/devices_by_id/device_id/2016/11/12/<filename>.json.gz
> From the documentation I figured the following queries were equivalent:
> select count(*) from `s3`.`/deviceid/xyz/2016/11/` ;
> +---------+
> | EXPR$0  |
> +---------+
> | 286049  |
> +---------+
> 1 row selected (10.351 seconds)
> select count(*) from `s3`.`/deviceid/` where dir0='xyz' and dir1='2016' and dir2='11'; But this latter query just hangs. There is no profile in the UI. I cntrl-c and get :
> +--+
> |  |
> +--+
> +--+
> No rows selected (1481.727 seconds)
> If I try to run an explain plan, that also hangs.
> There are a total of 13283 compressed json files in the 2016/11 s3 bucket. 
> The log doesn't show much information.
> If anyone can help with this please? I can provide more information as required. Hopefully this is not user error.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)