You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@drill.apache.org by Paul Pearcy <Pa...@blackboard.com> on 2015/01/24 02:55:41 UTC

Specify custom extension for JSON

Hello,
  I have a bunch of .gz that contain JSON that I am testing on a local on local filesystem. If I rename them to json.gz, they are recognized.

How can I configure drill to treat to treat them as compressed JSON?

I've tried adding a "format" to my storage config that looks like this:

    "gz": {
      "type": "json"
    }

But no luck:

0: jdbc:drill:zk=local> select distinct(schema) from dfs.`/drill/data/rawgz/activity/`;

Jan 23, 2015 8:49:58 PM org.eigenbase.sql.validate.SqlValidatorException <init>

SEVERE: org.eigenbase.sql.validate.SqlValidatorException: Table 'dfs./drill/data/rawgz/activity/' not found

Jan 23, 2015 8:49:58 PM org.eigenbase.util.EigenbaseException <init>

SEVERE: org.eigenbase.util.EigenbaseContextException: From line 1, column 30 to line 1, column 32: Table 'dfs./drill/data/rawgz/activity/' not found

Query failed: SqlValidatorException: Table 'dfs./drill/data/rawgz/activity/' not found


Thanks!
Paul
This email and any attachments may contain confidential and proprietary information of Blackboard that is for the sole use of the intended recipient. If you are not the intended recipient, disclosure, copying, re-distribution or other use of any of this information is strictly prohibited. Please immediately notify the sender and delete this transmission if you received this email in error.

Re: Specify custom extension for JSON

Posted by Paul Pearcy <Pa...@blackboard.com>.

Thanks Jacques. I gave it a shot, but still failed. I attached an updated
patch that should address.

Best Regards,
Paul




On 1/24/15, 3:24 PM, "Jacques Nadeau" <ja...@apache.org> wrote:

>This is being tracked here:
>https://issues.apache.org/jira/browse/DRILL-1545
>
>There is patch you can apply or we'll get it merged shortly.
>
>On Fri, Jan 23, 2015 at 5:55 PM, Paul Pearcy <Pa...@blackboard.com>
>wrote:
>
>> Hello,
>>   I have a bunch of .gz that contain JSON that I am testing on a local
>>on
>> local filesystem. If I rename them to json.gz, they are recognized.
>>
>> How can I configure drill to treat to treat them as compressed JSON?
>>
>> I've tried adding a "format" to my storage config that looks like this:
>>
>>     "gz": {
>>       "type": "json"
>>     }
>>
>> But no luck:
>>
>> 0: jdbc:drill:zk=local> select distinct(schema) from
>> dfs.`/drill/data/rawgz/activity/`;
>>
>> Jan 23, 2015 8:49:58 PM org.eigenbase.sql.validate.SqlValidatorException
>> <init>
>>
>> SEVERE: org.eigenbase.sql.validate.SqlValidatorException: Table
>> 'dfs./drill/data/rawgz/activity/' not found
>>
>> Jan 23, 2015 8:49:58 PM org.eigenbase.util.EigenbaseException <init>
>>
>> SEVERE: org.eigenbase.util.EigenbaseContextException: From line 1,
>>column
>> 30 to line 1, column 32: Table 'dfs./drill/data/rawgz/activity/' not
>>found
>>
>> Query failed: SqlValidatorException: Table
>> 'dfs./drill/data/rawgz/activity/' not found
>>
>>
>> Thanks!
>> Paul
>> This email and any attachments may contain confidential and proprietary
>> information of Blackboard that is for the sole use of the intended
>> recipient. If you are not the intended recipient, disclosure, copying,
>> re-distribution or other use of any of this information is strictly
>> prohibited. Please immediately notify the sender and delete this
>> transmission if you received this email in error.
>>

This email and any attachments may contain confidential and proprietary information of Blackboard that is for the sole use of the intended recipient. If you are not the intended recipient, disclosure, copying, re-distribution or other use of any of this information is strictly prohibited. Please immediately notify the sender and delete this transmission if you received this email in error.

Re: Specify custom extension for JSON

Posted by Jacques Nadeau <ja...@apache.org>.

This is being tracked here:
https://issues.apache.org/jira/browse/DRILL-1545

There is patch you can apply or we'll get it merged shortly.

On Fri, Jan 23, 2015 at 5:55 PM, Paul Pearcy <Pa...@blackboard.com>
wrote:

> Hello,
>   I have a bunch of .gz that contain JSON that I am testing on a local on
> local filesystem. If I rename them to json.gz, they are recognized.
>
> How can I configure drill to treat to treat them as compressed JSON?
>
> I've tried adding a "format" to my storage config that looks like this:
>
>     "gz": {
>       "type": "json"
>     }
>
> But no luck:
>
> 0: jdbc:drill:zk=local> select distinct(schema) from
> dfs.`/drill/data/rawgz/activity/`;
>
> Jan 23, 2015 8:49:58 PM org.eigenbase.sql.validate.SqlValidatorException
> <init>
>
> SEVERE: org.eigenbase.sql.validate.SqlValidatorException: Table
> 'dfs./drill/data/rawgz/activity/' not found
>
> Jan 23, 2015 8:49:58 PM org.eigenbase.util.EigenbaseException <init>
>
> SEVERE: org.eigenbase.util.EigenbaseContextException: From line 1, column
> 30 to line 1, column 32: Table 'dfs./drill/data/rawgz/activity/' not found
>
> Query failed: SqlValidatorException: Table
> 'dfs./drill/data/rawgz/activity/' not found
>
>
> Thanks!
> Paul
> This email and any attachments may contain confidential and proprietary
> information of Blackboard that is for the sole use of the intended
> recipient. If you are not the intended recipient, disclosure, copying,
> re-distribution or other use of any of this information is strictly
> prohibited. Please immediately notify the sender and delete this
> transmission if you received this email in error.
>