You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Sungwook Yoon <sy...@maprtech.com> on 2015/01/06 21:35:28 UTC
create table as default to parquet?
Hi
I am trying to save the query as csv
So, I am doing
create table as dfs.tmp.`/tmp.csv` select ..
It creates a parquet file.
Why did it not create csv file?
Thanks,
Sungwook
Re: create table as default to parquet?
Posted by Steven Phillips <sp...@maprtech.com>.
When creating the table, the path you provide becomes the directory that
holds the data. Extensions on directories don't generally have any meaning.
On Wed, Jan 7, 2015 at 8:56 AM, Andries Engelbrecht <
aengelbrecht@maprtech.com> wrote:
>
> storage plug in (if defined) < session level setting (if defined) < query
> (if defined)
>
> This way if the SP defines a default format for the workspace it is
> honored, unless specifically altered at session level, and if a specific
> query needs a specific format it can be defined at that instance by calling
> a specific suffix/format.
>
> —Andries
>
>
> On Jan 7, 2015, at 8:42 AM, Jason Altekruse <al...@gmail.com>
> wrote:
>
> > As we currently use file suffixes to determine file types on read, I
> think
> > it would make sense to have the same behavior on write (obviously with
> the
> > option to define overrides as users need them). Thoughts on the best user
> > experience here?
> >
> > -Jason Altekruse
> >
> > On Tue, Jan 6, 2015 at 1:01 PM, Sungwook Yoon <sy...@maprtech.com>
> wrote:
> >
> >> Hey... Abdel,
> >>
> >> Thanks,
> >> It works
> >>
> >> Sungwook
> >>
> >>
> >> On Tue, Jan 6, 2015 at 12:40 PM, Abdel Hakim Deneche <
> >> adeneche@maprtech.com>
> >> wrote:
> >>
> >>> According to the Wiki
> >>> <
> >>>
> >>
> https://cwiki.apache.org/confluence/display/DRILL/CREATE+TABLE+AS+%28CTAS%29+Command
> >>>>
> >>> you need to call:
> >>>
> >>> alter session set `store.format`='json';
> >>>
> >>> to change the storage format.
> >>>
> >>>
> >>>
> >>> On Tue, Jan 6, 2015 at 12:35 PM, Sungwook Yoon <sy...@maprtech.com>
> >> wrote:
> >>>
> >>>> Hi
> >>>>
> >>>> I am trying to save the query as csv
> >>>>
> >>>> So, I am doing
> >>>>
> >>>> create table as dfs.tmp.`/tmp.csv` select ..
> >>>>
> >>>> It creates a parquet file.
> >>>> Why did it not create csv file?
> >>>>
> >>>> Thanks,
> >>>>
> >>>> Sungwook
> >>>>
> >>>
> >>
>
>
--
Steven Phillips
Software Engineer
mapr.com
Re: create table as default to parquet?
Posted by Andries Engelbrecht <ae...@maprtech.com>.
storage plug in (if defined) < session level setting (if defined) < query (if defined)
This way if the SP defines a default format for the workspace it is honored, unless specifically altered at session level, and if a specific query needs a specific format it can be defined at that instance by calling a specific suffix/format.
—Andries
On Jan 7, 2015, at 8:42 AM, Jason Altekruse <al...@gmail.com> wrote:
> As we currently use file suffixes to determine file types on read, I think
> it would make sense to have the same behavior on write (obviously with the
> option to define overrides as users need them). Thoughts on the best user
> experience here?
>
> -Jason Altekruse
>
> On Tue, Jan 6, 2015 at 1:01 PM, Sungwook Yoon <sy...@maprtech.com> wrote:
>
>> Hey... Abdel,
>>
>> Thanks,
>> It works
>>
>> Sungwook
>>
>>
>> On Tue, Jan 6, 2015 at 12:40 PM, Abdel Hakim Deneche <
>> adeneche@maprtech.com>
>> wrote:
>>
>>> According to the Wiki
>>> <
>>>
>> https://cwiki.apache.org/confluence/display/DRILL/CREATE+TABLE+AS+%28CTAS%29+Command
>>>>
>>> you need to call:
>>>
>>> alter session set `store.format`='json';
>>>
>>> to change the storage format.
>>>
>>>
>>>
>>> On Tue, Jan 6, 2015 at 12:35 PM, Sungwook Yoon <sy...@maprtech.com>
>> wrote:
>>>
>>>> Hi
>>>>
>>>> I am trying to save the query as csv
>>>>
>>>> So, I am doing
>>>>
>>>> create table as dfs.tmp.`/tmp.csv` select ..
>>>>
>>>> It creates a parquet file.
>>>> Why did it not create csv file?
>>>>
>>>> Thanks,
>>>>
>>>> Sungwook
>>>>
>>>
>>
Re: create table as default to parquet?
Posted by Sungwook Yoon <sy...@maprtech.com>.
Jason, I agree.
Customer Corning is okay with the current interface, but it is not the most
natural interface at this point.
Automatically recognizing the format with file extension would be more
consistent.
Sungwook
On Wed, Jan 7, 2015 at 8:42 AM, Jason Altekruse <al...@gmail.com>
wrote:
> As we currently use file suffixes to determine file types on read, I think
> it would make sense to have the same behavior on write (obviously with the
> option to define overrides as users need them). Thoughts on the best user
> experience here?
>
> -Jason Altekruse
>
> On Tue, Jan 6, 2015 at 1:01 PM, Sungwook Yoon <sy...@maprtech.com> wrote:
>
> > Hey... Abdel,
> >
> > Thanks,
> > It works
> >
> > Sungwook
> >
> >
> > On Tue, Jan 6, 2015 at 12:40 PM, Abdel Hakim Deneche <
> > adeneche@maprtech.com>
> > wrote:
> >
> > > According to the Wiki
> > > <
> > >
> >
> https://cwiki.apache.org/confluence/display/DRILL/CREATE+TABLE+AS+%28CTAS%29+Command
> > > >
> > > you need to call:
> > >
> > > alter session set `store.format`='json';
> > >
> > > to change the storage format.
> > >
> > >
> > >
> > > On Tue, Jan 6, 2015 at 12:35 PM, Sungwook Yoon <sy...@maprtech.com>
> > wrote:
> > >
> > > > Hi
> > > >
> > > > I am trying to save the query as csv
> > > >
> > > > So, I am doing
> > > >
> > > > create table as dfs.tmp.`/tmp.csv` select ..
> > > >
> > > > It creates a parquet file.
> > > > Why did it not create csv file?
> > > >
> > > > Thanks,
> > > >
> > > > Sungwook
> > > >
> > >
> >
>
Re: create table as default to parquet?
Posted by Jason Altekruse <al...@gmail.com>.
As we currently use file suffixes to determine file types on read, I think
it would make sense to have the same behavior on write (obviously with the
option to define overrides as users need them). Thoughts on the best user
experience here?
-Jason Altekruse
On Tue, Jan 6, 2015 at 1:01 PM, Sungwook Yoon <sy...@maprtech.com> wrote:
> Hey... Abdel,
>
> Thanks,
> It works
>
> Sungwook
>
>
> On Tue, Jan 6, 2015 at 12:40 PM, Abdel Hakim Deneche <
> adeneche@maprtech.com>
> wrote:
>
> > According to the Wiki
> > <
> >
> https://cwiki.apache.org/confluence/display/DRILL/CREATE+TABLE+AS+%28CTAS%29+Command
> > >
> > you need to call:
> >
> > alter session set `store.format`='json';
> >
> > to change the storage format.
> >
> >
> >
> > On Tue, Jan 6, 2015 at 12:35 PM, Sungwook Yoon <sy...@maprtech.com>
> wrote:
> >
> > > Hi
> > >
> > > I am trying to save the query as csv
> > >
> > > So, I am doing
> > >
> > > create table as dfs.tmp.`/tmp.csv` select ..
> > >
> > > It creates a parquet file.
> > > Why did it not create csv file?
> > >
> > > Thanks,
> > >
> > > Sungwook
> > >
> >
>
Re: create table as default to parquet?
Posted by Sungwook Yoon <sy...@maprtech.com>.
Hey... Abdel,
Thanks,
It works
Sungwook
On Tue, Jan 6, 2015 at 12:40 PM, Abdel Hakim Deneche <ad...@maprtech.com>
wrote:
> According to the Wiki
> <
> https://cwiki.apache.org/confluence/display/DRILL/CREATE+TABLE+AS+%28CTAS%29+Command
> >
> you need to call:
>
> alter session set `store.format`='json';
>
> to change the storage format.
>
>
>
> On Tue, Jan 6, 2015 at 12:35 PM, Sungwook Yoon <sy...@maprtech.com> wrote:
>
> > Hi
> >
> > I am trying to save the query as csv
> >
> > So, I am doing
> >
> > create table as dfs.tmp.`/tmp.csv` select ..
> >
> > It creates a parquet file.
> > Why did it not create csv file?
> >
> > Thanks,
> >
> > Sungwook
> >
>
Re: create table as default to parquet?
Posted by Abdel Hakim Deneche <ad...@maprtech.com>.
According to the Wiki
<https://cwiki.apache.org/confluence/display/DRILL/CREATE+TABLE+AS+%28CTAS%29+Command>
you need to call:
alter session set `store.format`='json';
to change the storage format.
On Tue, Jan 6, 2015 at 12:35 PM, Sungwook Yoon <sy...@maprtech.com> wrote:
> Hi
>
> I am trying to save the query as csv
>
> So, I am doing
>
> create table as dfs.tmp.`/tmp.csv` select ..
>
> It creates a parquet file.
> Why did it not create csv file?
>
> Thanks,
>
> Sungwook
>