You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Paul Pearcy <Pa...@blackboard.com> on 2014/12/17 21:03:43 UTC
Query gz compressed JSON files with Apache Drill
Hello,
New to drill, so hopefully am not missing something obvious. I have
Apache Drill querying uncompressed JSON files with no problem, but am
struggling with gz compressed JSON archives.
My understanding is that Drill uses the Hadoop file connector which I
believed had the ability to handle gz files, but it seems that that
Drill's JSON querying capabilities are always locked to .json files.
I've tried doing something like this:
"formats": {
"gz": {
"type": "json"
}
}
However, receive a file not found error. Also tried this:
"formats": {
"json": {
"type": "json",
"extensions": [
"gz"
]
}
}
Which results in an "invalid JSON mapping" error.
Thanks!
Paul
This email and any attachments may contain confidential and proprietary information of Blackboard that is for the sole use of the intended recipient. If you are not the intended recipient, disclosure, copying, re-distribution or other use of any of this information is strictly prohibited. Please immediately notify the sender and delete this transmission if you received this email in error.
Re: Query gz compressed JSON files with Apache Drill
Posted by Parth Chandra <pc...@maprtech.com>.
Unfortunately no. It's still in testing. We'll be pushing it into the 0.8.0 snapshot in a bit.
> On Dec 17, 2014, at 7:51 PM, Tomer Shiran <ts...@gmail.com> wrote:
>
> Will this be in 0.7 which is being released now?
>
>
>
>> On Dec 17, 2014, at 12:05 PM, Steven Phillips <sp...@maprtech.com> wrote:
>>
>> There is a bug that is preventing reading compressed json files. There is a
>> fix currently in the works and should be available soon. I don't know if
>> there is a jira filed for this yet.
>>
>> On Wed, Dec 17, 2014 at 12:03 PM, Paul Pearcy <Pa...@blackboard.com>
>> wrote:
>>>
>>> Hello,
>>> New to drill, so hopefully am not missing something obvious. I have
>>> Apache Drill querying uncompressed JSON files with no problem, but am
>>> struggling with gz compressed JSON archives.
>>>
>>> My understanding is that Drill uses the Hadoop file connector which I
>>> believed had the ability to handle gz files, but it seems that that
>>> Drill's JSON querying capabilities are always locked to .json files.
>>> I've tried doing something like this:
>>>
>>> "formats": {
>>> "gz": {
>>> "type": "json"
>>> }
>>> }
>>>
>>> However, receive a file not found error. Also tried this:
>>>
>>> "formats": {
>>> "json": {
>>> "type": "json",
>>> "extensions": [
>>> "gz"
>>> ]
>>> }
>>> }
>>>
>>> Which results in an "invalid JSON mapping" error.
>>>
>>> Thanks!
>>> Paul
>>>
>>> This email and any attachments may contain confidential and proprietary
>>> information of Blackboard that is for the sole use of the intended
>>> recipient. If you are not the intended recipient, disclosure, copying,
>>> re-distribution or other use of any of this information is strictly
>>> prohibited. Please immediately notify the sender and delete this
>>> transmission if you received this email in error.
>>
>>
>> --
>> Steven Phillips
>> Software Engineer
>>
>> mapr.com
Re: Query gz compressed JSON files with Apache Drill
Posted by Tomer Shiran <ts...@gmail.com>.
Will this be in 0.7 which is being released now?
> On Dec 17, 2014, at 12:05 PM, Steven Phillips <sp...@maprtech.com> wrote:
>
> There is a bug that is preventing reading compressed json files. There is a
> fix currently in the works and should be available soon. I don't know if
> there is a jira filed for this yet.
>
> On Wed, Dec 17, 2014 at 12:03 PM, Paul Pearcy <Pa...@blackboard.com>
> wrote:
>>
>> Hello,
>> New to drill, so hopefully am not missing something obvious. I have
>> Apache Drill querying uncompressed JSON files with no problem, but am
>> struggling with gz compressed JSON archives.
>>
>> My understanding is that Drill uses the Hadoop file connector which I
>> believed had the ability to handle gz files, but it seems that that
>> Drill's JSON querying capabilities are always locked to .json files.
>> I've tried doing something like this:
>>
>> "formats": {
>> "gz": {
>> "type": "json"
>> }
>> }
>>
>> However, receive a file not found error. Also tried this:
>>
>> "formats": {
>> "json": {
>> "type": "json",
>> "extensions": [
>> "gz"
>> ]
>> }
>> }
>>
>> Which results in an "invalid JSON mapping" error.
>>
>> Thanks!
>> Paul
>>
>> This email and any attachments may contain confidential and proprietary
>> information of Blackboard that is for the sole use of the intended
>> recipient. If you are not the intended recipient, disclosure, copying,
>> re-distribution or other use of any of this information is strictly
>> prohibited. Please immediately notify the sender and delete this
>> transmission if you received this email in error.
>
>
> --
> Steven Phillips
> Software Engineer
>
> mapr.com
Re: Query gz compressed JSON files with Apache Drill
Posted by Steven Phillips <sp...@maprtech.com>.
There is a bug that is preventing reading compressed json files. There is a
fix currently in the works and should be available soon. I don't know if
there is a jira filed for this yet.
On Wed, Dec 17, 2014 at 12:03 PM, Paul Pearcy <Pa...@blackboard.com>
wrote:
>
> Hello,
> New to drill, so hopefully am not missing something obvious. I have
> Apache Drill querying uncompressed JSON files with no problem, but am
> struggling with gz compressed JSON archives.
>
> My understanding is that Drill uses the Hadoop file connector which I
> believed had the ability to handle gz files, but it seems that that
> Drill's JSON querying capabilities are always locked to .json files.
> I've tried doing something like this:
>
> "formats": {
> "gz": {
> "type": "json"
> }
> }
>
> However, receive a file not found error. Also tried this:
>
> "formats": {
> "json": {
> "type": "json",
> "extensions": [
> "gz"
> ]
> }
> }
>
> Which results in an "invalid JSON mapping" error.
>
> Thanks!
> Paul
>
> This email and any attachments may contain confidential and proprietary
> information of Blackboard that is for the sole use of the intended
> recipient. If you are not the intended recipient, disclosure, copying,
> re-distribution or other use of any of this information is strictly
> prohibited. Please immediately notify the sender and delete this
> transmission if you received this email in error.
>
--
Steven Phillips
Software Engineer
mapr.com