Issue Drill querying S3 recursive

[Magesh Waran <ma...@gmail.com>]

Hi Team,

I am trying to query a file under folder 't/atms-csv.csv' which I can
successfully do it.

[image: Inline image 1]


There is another file in that location which as additional data from
another day (both file columnmodel). when I try query recursively using
folder name 't' which shows below error
[image: Inline image 2]
*Tried queries (All doesnt worked out):*
1. select * from `t/`;
2. select * from `/t/`;
3. select * from dfs.`t/`;
4. select * from dfs`/t/`;
5. select * from fs.`t/`;
6. select * from fs`/t/`;


I am using below versions,
*Drill - 1.6.0*
*S3 protool: S3n (also tried S3, S3a)*
*Sample Query String:*
 {
 "connection": "s3a://xxxxx-bucket",
  "config": {
    "fs.s3a.access.key": "xxxxxxxxxxxxxxxxxxxxxxxxxxxx",
    "fs.s3a.secret.key": "xxxxxxxxxxxxxxxxxxxxxxxxxxxx"
  },
  "workspaces": {
    "root": {
      "location": "/",
      "writable": true,
      "defaultInputFormat": null
    }
}


Thanks & Regards
Mageshwaran M
Contact: +91-9944667832

Re: Issue Drill querying S3 recursive

[Andries Engelbrecht <ae...@mapr.com>]

You may want to upgrade to the latest Drill version (1.9).


Also what is the storage plugin called dfs, fs,...?


If it is called dfs then you can thry the following.


use dfs;

show files;


This will show you the files and directories in the plugin root, you can navigate from there.


If it shows you t as a directory you can then see which files are in t;


show files in `/t`;


See if the csv file shows up, if so this should work.


select * from dfs.`/t`;



--Andries

________________________________
From: Magesh Waran <ma...@gmail.com>
Sent: Thursday, January 12, 2017 6:31:22 AM
To: user@drill.apache.org
Subject: Issue Drill querying S3 recursive

Hi Team,

I am trying to query a file under folder 't/atms-csv.csv' which I can successfully do it.

[Inline image 1]


There is another file in that location which as additional data from another day (both file columnmodel). when I try query recursively using folder name 't' which shows below error
[Inline image 2]
Tried queries (All doesnt worked out):
1. select * from `t/`;
2. select * from `/t/`;
3. select * from dfs.`t/`;
4. select * from dfs`/t/`;
5. select * from fs.`t/`;
6. select * from fs`/t/`;


I am using below versions,
Drill - 1.6.0
S3 protool: S3n (also tried S3, S3a)
Sample Query String:
 {
 "connection": "s3a://xxxxx-bucket",
  "config": {
    "fs.s3a.access.key": "xxxxxxxxxxxxxxxxxxxxxxxxxxxx",
    "fs.s3a.secret.key": "xxxxxxxxxxxxxxxxxxxxxxxxxxxx"
  },
  "workspaces": {
    "root": {
      "location": "/",
      "writable": true,
      "defaultInputFormat": null
    }
}


Thanks & Regards
Mageshwaran M
Contact: +91-9944667832