You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Scott Kinney <sc...@stem.com> on 2016/05/31 20:26:17 UTC
Do i need hadoop installed to use dfs storage?
I'm trying to test running drill on gz json files in s3 and i keep getting.
VALIDATION ERROR: From line 1, column 15 to line 1, column 17: Table 's3_file.gz' not found
I downloaded the file and unzipped it, setup a new storage plugin to point to the local file
{
"type": "file",
"enabled": true,
"connection": "file:///tmp/data/",
"config": null,
"workspaces": {
"root": {
"location": "/",
"writable": false,
"defaultInputFormat": "json"
},
...
And i get the same thing.
I do not have hadoop installed. do i need it?
Thanks,
________________________________
Scott Kinney | DevOps
stem <http://www.stem.com/> | m 510.282.1299
100 Rollins Road, Millbrae, California 94030
This e-mail and/or any attachments contain Stem, Inc. confidential and proprietary information and material for the sole use of the intended recipient(s). Any review, use or distribution that has not been expressly authorized by Stem, Inc. is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. Thank you.
Re: Do i need hadoop installed to use dfs storage?
Posted by Scott Kinney <sc...@stem.com>.
There is something wrong with my storage plugin config because it works when i use the default dfs storage plugin and query like.
SELECT * FROM dfs.`/path/to/jsonfile.json`;
probably
...
"connection": "file:///tmp/data/",
...
i left the three /
________________________________
Scott Kinney | DevOps
stem | m 510.282.1299
100 Rollins Road, Millbrae, California 94030
This e-mail and/or any attachments contain Stem, Inc. confidential and proprietary information and material for the sole use of the intended recipient(s). Any review, use or distribution that has not been expressly authorized by Stem, Inc. is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. Thank you.
________________________________________
From: Nathan Griffith <ng...@dremio.com>
Sent: Tuesday, May 31, 2016 5:23 PM
To: user@drill.apache.org
Subject: Re: Do i need hadoop installed to use dfs storage?
Hi Scott,
You definitely don't need to have Hadoop to query your local file system.
Could you list the exact command to Drill that gave you this error?
Best,
Nathan Griffith
Technical Writer
Dremio
On Tue, May 31, 2016 at 1:26 PM, Scott Kinney <sc...@stem.com> wrote:
>
> I'm trying to test running drill on gz json files in s3 and i keep getting.
>
>
> VALIDATION ERROR: From line 1, column 15 to line 1, column 17: Table
> 's3_file.gz' not found
>
>
> I downloaded the file and unzipped it, setup a new storage plugin to point
> to the local file
>
>
> {
> "type": "file",
> "enabled": true,
> "connection": "file:///tmp/data/",
> "config": null,
> "workspaces": {
> "root": {
> "location": "/",
> "writable": false,
> "defaultInputFormat": "json"
> },
> ...
>
> And i get the same thing.
> I do not have hadoop installed. do i need it?
>
> Thanks,
>
>
>
> ________________________________
> Scott Kinney | DevOps
> stem <http://www.stem.com/> | m 510.282.1299
> 100 Rollins Road, Millbrae, California 94030
>
> This e-mail and/or any attachments contain Stem, Inc. confidential and
> proprietary information and material for the sole use of the intended
> recipient(s). Any review, use or distribution that has not been expressly
> authorized by Stem, Inc. is strictly prohibited. If you are not the
> intended recipient, please contact the sender and delete all copies. Thank
> you.
>
Re: Do i need hadoop installed to use dfs storage?
Posted by Nathan Griffith <ng...@dremio.com>.
Hi Scott,
You definitely don't need to have Hadoop to query your local file system.
Could you list the exact command to Drill that gave you this error?
Best,
Nathan Griffith
Technical Writer
Dremio
On Tue, May 31, 2016 at 1:26 PM, Scott Kinney <sc...@stem.com> wrote:
>
> I'm trying to test running drill on gz json files in s3 and i keep getting.
>
>
> VALIDATION ERROR: From line 1, column 15 to line 1, column 17: Table
> 's3_file.gz' not found
>
>
> I downloaded the file and unzipped it, setup a new storage plugin to point
> to the local file
>
>
> {
> "type": "file",
> "enabled": true,
> "connection": "file:///tmp/data/",
> "config": null,
> "workspaces": {
> "root": {
> "location": "/",
> "writable": false,
> "defaultInputFormat": "json"
> },
> ...
>
> And i get the same thing.
> I do not have hadoop installed. do i need it?
>
> Thanks,
>
>
>
> ________________________________
> Scott Kinney | DevOps
> stem <http://www.stem.com/> | m 510.282.1299
> 100 Rollins Road, Millbrae, California 94030
>
> This e-mail and/or any attachments contain Stem, Inc. confidential and
> proprietary information and material for the sole use of the intended
> recipient(s). Any review, use or distribution that has not been expressly
> authorized by Stem, Inc. is strictly prohibited. If you are not the
> intended recipient, please contact the sender and delete all copies. Thank
> you.
>