You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by ДП <di...@yandex.ru> on 2014/12/11 08:12:39 UTC
Drill plugin for zipped files
Hi!
Apache Drill is most interesting project!
I think, Apache Drill is best project for view and analyze logs by sql-developers and managers!
But I have some disappointment. =(
Most logs are stored in zip, gzip, tar or other archive and packed formats.
How I can to config dfs storage to use unzip/gunzip command before get data?
Or can you add plugin for zipped files?
---------------
Best regards,
Dima Pl
Re: Drill plugin for zipped files
Posted by Jason Altekruse <al...@gmail.com>.
Is this feature documented? A quick search of the drill wiki for
'compressed', 'zip' and 'gzip' does not seem to show any useful results.
- Jason
On Fri, Dec 12, 2014 at 10:36 AM, Steven Phillips <sp...@maprtech.com>
wrote:
>
> You want to keep the .gz extension:
>
> 0: jdbc:drill:zk=local> select * from dfs.`/drill/nation.csv.gz` limit 2;
> +------------+
> | columns |
> +------------+
> | ["0","ALGERIA","0"," haggle. carefully final deposits detect slyly agai"]
> |
> | ["1","ARGENTINA","1","al foxes promise slyly according to the regular
> accounts. bold requests alon"] |
> +------------+
>
> bzip2 is also supported:
>
> 0: jdbc:drill:zk=local> select * from dfs.`/drill/nation.csv.bz2` limit 2;
> +------------+
> | columns |
> +------------+
> | ["0","ALGERIA","0"," haggle. carefully final deposits detect slyly agai"]
> |
> | ["1","ARGENTINA","1","al foxes promise slyly according to the regular
> accounts. bold requests alon"] |
> +------------+
>
> On Fri, Dec 12, 2014 at 6:01 AM, Hmxxyy <hm...@gmail.com> wrote:
> >
> > Could you please show an example for that? Especially how to define the
> > file suffix, should I renamed the data files from .csv.gz to .csv? Does
> it
> > support bz2 as well?
> >
> > Thanks.
> >
> > Sent from my iPhone
> >
> > > On Dec 12, 2014, at 5:49 AM, Jim Scott <js...@maprtech.com> wrote:
> > >
> > > With csv files (as an example) it will automatically decompress them.
> > >
> > >> On Thu, Dec 11, 2014 at 1:12 AM, ДП <di...@yandex.ru> wrote:
> > >>
> > >> Hi!
> > >> Apache Drill is most interesting project!
> > >> I think, Apache Drill is best project for view and analyze logs by
> > >> sql-developers and managers!
> > >> But I have some disappointment. =(
> > >> Most logs are stored in zip, gzip, tar or other archive and packed
> > formats.
> > >> How I can to config dfs storage to use unzip/gunzip command before get
> > >> data?
> > >> Or can you add plugin for zipped files?
> > >> ---------------
> > >> Best regards,
> > >> Dima Pl
> > >
> > >
> > > --
> > > *Jim Scott*
> > > Director, Enterprise Strategy & Architecture
> > > +1 (347) 746-9281
> > >
> > > <http://www.mapr.com/>
> > > [image: MapR Technologies] <http://www.mapr.com>
> >
>
>
> --
> Steven Phillips
> Software Engineer
>
> mapr.com
>
Re: Drill plugin for zipped files
Posted by Steven Phillips <sp...@maprtech.com>.
You want to keep the .gz extension:
0: jdbc:drill:zk=local> select * from dfs.`/drill/nation.csv.gz` limit 2;
+------------+
| columns |
+------------+
| ["0","ALGERIA","0"," haggle. carefully final deposits detect slyly agai"]
|
| ["1","ARGENTINA","1","al foxes promise slyly according to the regular
accounts. bold requests alon"] |
+------------+
bzip2 is also supported:
0: jdbc:drill:zk=local> select * from dfs.`/drill/nation.csv.bz2` limit 2;
+------------+
| columns |
+------------+
| ["0","ALGERIA","0"," haggle. carefully final deposits detect slyly agai"]
|
| ["1","ARGENTINA","1","al foxes promise slyly according to the regular
accounts. bold requests alon"] |
+------------+
On Fri, Dec 12, 2014 at 6:01 AM, Hmxxyy <hm...@gmail.com> wrote:
>
> Could you please show an example for that? Especially how to define the
> file suffix, should I renamed the data files from .csv.gz to .csv? Does it
> support bz2 as well?
>
> Thanks.
>
> Sent from my iPhone
>
> > On Dec 12, 2014, at 5:49 AM, Jim Scott <js...@maprtech.com> wrote:
> >
> > With csv files (as an example) it will automatically decompress them.
> >
> >> On Thu, Dec 11, 2014 at 1:12 AM, ДП <di...@yandex.ru> wrote:
> >>
> >> Hi!
> >> Apache Drill is most interesting project!
> >> I think, Apache Drill is best project for view and analyze logs by
> >> sql-developers and managers!
> >> But I have some disappointment. =(
> >> Most logs are stored in zip, gzip, tar or other archive and packed
> formats.
> >> How I can to config dfs storage to use unzip/gunzip command before get
> >> data?
> >> Or can you add plugin for zipped files?
> >> ---------------
> >> Best regards,
> >> Dima Pl
> >
> >
> > --
> > *Jim Scott*
> > Director, Enterprise Strategy & Architecture
> > +1 (347) 746-9281
> >
> > <http://www.mapr.com/>
> > [image: MapR Technologies] <http://www.mapr.com>
>
--
Steven Phillips
Software Engineer
mapr.com
Re: Drill plugin for zipped files
Posted by Steven Phillips <sp...@maprtech.com>.
You want to keep the .gz extension:
0: jdbc:drill:zk=local> select * from dfs.`/drill/nation.csv.gz` limit 2;
+------------+
| columns |
+------------+
| ["0","ALGERIA","0"," haggle. carefully final deposits detect slyly agai"]
|
| ["1","ARGENTINA","1","al foxes promise slyly according to the regular
accounts. bold requests alon"] |
+------------+
bzip2 is also supported:
0: jdbc:drill:zk=local> select * from dfs.`/drill/nation.csv.bz2` limit 2;
+------------+
| columns |
+------------+
| ["0","ALGERIA","0"," haggle. carefully final deposits detect slyly agai"]
|
| ["1","ARGENTINA","1","al foxes promise slyly according to the regular
accounts. bold requests alon"] |
+------------+
On Fri, Dec 12, 2014 at 6:01 AM, Hmxxyy <hm...@gmail.com> wrote:
>
> Could you please show an example for that? Especially how to define the
> file suffix, should I renamed the data files from .csv.gz to .csv? Does it
> support bz2 as well?
>
> Thanks.
>
> Sent from my iPhone
>
> > On Dec 12, 2014, at 5:49 AM, Jim Scott <js...@maprtech.com> wrote:
> >
> > With csv files (as an example) it will automatically decompress them.
> >
> >> On Thu, Dec 11, 2014 at 1:12 AM, ДП <di...@yandex.ru> wrote:
> >>
> >> Hi!
> >> Apache Drill is most interesting project!
> >> I think, Apache Drill is best project for view and analyze logs by
> >> sql-developers and managers!
> >> But I have some disappointment. =(
> >> Most logs are stored in zip, gzip, tar or other archive and packed
> formats.
> >> How I can to config dfs storage to use unzip/gunzip command before get
> >> data?
> >> Or can you add plugin for zipped files?
> >> ---------------
> >> Best regards,
> >> Dima Pl
> >
> >
> > --
> > *Jim Scott*
> > Director, Enterprise Strategy & Architecture
> > +1 (347) 746-9281
> >
> > <http://www.mapr.com/>
> > [image: MapR Technologies] <http://www.mapr.com>
>
--
Steven Phillips
Software Engineer
mapr.com
Re: Drill plugin for zipped files
Posted by Hmxxyy <hm...@gmail.com>.
Could you please show an example for that? Especially how to define the file suffix, should I renamed the data files from .csv.gz to .csv? Does it support bz2 as well?
Thanks.
Sent from my iPhone
> On Dec 12, 2014, at 5:49 AM, Jim Scott <js...@maprtech.com> wrote:
>
> With csv files (as an example) it will automatically decompress them.
>
>> On Thu, Dec 11, 2014 at 1:12 AM, ДП <di...@yandex.ru> wrote:
>>
>> Hi!
>> Apache Drill is most interesting project!
>> I think, Apache Drill is best project for view and analyze logs by
>> sql-developers and managers!
>> But I have some disappointment. =(
>> Most logs are stored in zip, gzip, tar or other archive and packed formats.
>> How I can to config dfs storage to use unzip/gunzip command before get
>> data?
>> Or can you add plugin for zipped files?
>> ---------------
>> Best regards,
>> Dima Pl
>
>
> --
> *Jim Scott*
> Director, Enterprise Strategy & Architecture
> +1 (347) 746-9281
>
> <http://www.mapr.com/>
> [image: MapR Technologies] <http://www.mapr.com>
Re: Drill plugin for zipped files
Posted by Hmxxyy <hm...@gmail.com>.
Could you please show an example for that? Especially how to define the file suffix, should I renamed the data files from .csv.gz to .csv? Does it support bz2 as well?
Thanks.
Sent from my iPhone
> On Dec 12, 2014, at 5:49 AM, Jim Scott <js...@maprtech.com> wrote:
>
> With csv files (as an example) it will automatically decompress them.
>
>> On Thu, Dec 11, 2014 at 1:12 AM, ДП <di...@yandex.ru> wrote:
>>
>> Hi!
>> Apache Drill is most interesting project!
>> I think, Apache Drill is best project for view and analyze logs by
>> sql-developers and managers!
>> But I have some disappointment. =(
>> Most logs are stored in zip, gzip, tar or other archive and packed formats.
>> How I can to config dfs storage to use unzip/gunzip command before get
>> data?
>> Or can you add plugin for zipped files?
>> ---------------
>> Best regards,
>> Dima Pl
>
>
> --
> *Jim Scott*
> Director, Enterprise Strategy & Architecture
> +1 (347) 746-9281
>
> <http://www.mapr.com/>
> [image: MapR Technologies] <http://www.mapr.com>
Re: Drill plugin for zipped files
Posted by Jim Scott <js...@maprtech.com>.
With csv files (as an example) it will automatically decompress them.
On Thu, Dec 11, 2014 at 1:12 AM, ДП <di...@yandex.ru> wrote:
>
> Hi!
> Apache Drill is most interesting project!
> I think, Apache Drill is best project for view and analyze logs by
> sql-developers and managers!
> But I have some disappointment. =(
> Most logs are stored in zip, gzip, tar or other archive and packed formats.
> How I can to config dfs storage to use unzip/gunzip command before get
> data?
> Or can you add plugin for zipped files?
> ---------------
> Best regards,
> Dima Pl
>
--
*Jim Scott*
Director, Enterprise Strategy & Architecture
+1 (347) 746-9281
<http://www.mapr.com/>
[image: MapR Technologies] <http://www.mapr.com>
Re: Drill plugin for zipped files
Posted by Jim Scott <js...@maprtech.com>.
With csv files (as an example) it will automatically decompress them.
On Thu, Dec 11, 2014 at 1:12 AM, ДП <di...@yandex.ru> wrote:
>
> Hi!
> Apache Drill is most interesting project!
> I think, Apache Drill is best project for view and analyze logs by
> sql-developers and managers!
> But I have some disappointment. =(
> Most logs are stored in zip, gzip, tar or other archive and packed formats.
> How I can to config dfs storage to use unzip/gunzip command before get
> data?
> Or can you add plugin for zipped files?
> ---------------
> Best regards,
> Dima Pl
>
--
*Jim Scott*
Director, Enterprise Strategy & Architecture
+1 (347) 746-9281
<http://www.mapr.com/>
[image: MapR Technologies] <http://www.mapr.com>