You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by rahul challapalli <ch...@gmail.com> on 2015/02/24 00:29:18 UTC
Using CTAS with nested structures
Hi,
I want to able to read from a nested json file, cast individual fields, and
still be able to maintain the nested structure. If this is possible, how do
I compose my CTAS query?
Example :
{
"map" : {"col1":12.2, "col2":25.5}
}
The below query generates a flat parquet, but I still want to maintain the
nestedness
select cast(d.map.col1 as decimal(5,2)) , cast(d.map.col2 as double)) from
`data.json`;
I am looking for something on the below lines :
select cast(m1 as map(col1:decimal, col2:double)) from `data.json`;
- Rahul
Re: Using CTAS with nested structures
Posted by rahul challapalli <ch...@gmail.com>.
I will raise an enhancement request for this. Thank You.
- Rahul
On Mon, Feb 23, 2015 at 3:35 PM, Adam Gilmore <dr...@gmail.com> wrote:
> I submitted a patch for reading all JSON numbers as doubles; however, it'd
> probably be nice to extend that to specify a default to read as anything.
>
> Something like ...
>
> alter session set `store.json.read_numbers_as` = 'DECIMAL(5,2)';
>
> would be useful.
>
> On Tue, Feb 24, 2015 at 9:32 AM, Steven Phillips <sp...@maprtech.com>
> wrote:
>
> > I don't think this is supported currently.
> >
> > On Mon, Feb 23, 2015 at 3:29 PM, rahul challapalli <
> > challapallirahul@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > I want to able to read from a nested json file, cast individual fields,
> > and
> > > still be able to maintain the nested structure. If this is possible,
> how
> > do
> > > I compose my CTAS query?
> > >
> > > Example :
> > >
> > > {
> > > "map" : {"col1":12.2, "col2":25.5}
> > > }
> > >
> > > The below query generates a flat parquet, but I still want to maintain
> > the
> > > nestedness
> > >
> > > select cast(d.map.col1 as decimal(5,2)) , cast(d.map.col2 as double))
> > from
> > > `data.json`;
> > >
> > >
> > > I am looking for something on the below lines :
> > >
> > > select cast(m1 as map(col1:decimal, col2:double)) from `data.json`;
> > >
> > > - Rahul
> > >
> >
> >
> >
> > --
> > Steven Phillips
> > Software Engineer
> >
> > mapr.com
> >
>
Re: Using CTAS with nested structures
Posted by rahul challapalli <ch...@gmail.com>.
I will raise an enhancement request for this. Thank You.
- Rahul
On Mon, Feb 23, 2015 at 3:35 PM, Adam Gilmore <dr...@gmail.com> wrote:
> I submitted a patch for reading all JSON numbers as doubles; however, it'd
> probably be nice to extend that to specify a default to read as anything.
>
> Something like ...
>
> alter session set `store.json.read_numbers_as` = 'DECIMAL(5,2)';
>
> would be useful.
>
> On Tue, Feb 24, 2015 at 9:32 AM, Steven Phillips <sp...@maprtech.com>
> wrote:
>
> > I don't think this is supported currently.
> >
> > On Mon, Feb 23, 2015 at 3:29 PM, rahul challapalli <
> > challapallirahul@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > I want to able to read from a nested json file, cast individual fields,
> > and
> > > still be able to maintain the nested structure. If this is possible,
> how
> > do
> > > I compose my CTAS query?
> > >
> > > Example :
> > >
> > > {
> > > "map" : {"col1":12.2, "col2":25.5}
> > > }
> > >
> > > The below query generates a flat parquet, but I still want to maintain
> > the
> > > nestedness
> > >
> > > select cast(d.map.col1 as decimal(5,2)) , cast(d.map.col2 as double))
> > from
> > > `data.json`;
> > >
> > >
> > > I am looking for something on the below lines :
> > >
> > > select cast(m1 as map(col1:decimal, col2:double)) from `data.json`;
> > >
> > > - Rahul
> > >
> >
> >
> >
> > --
> > Steven Phillips
> > Software Engineer
> >
> > mapr.com
> >
>
Re: Using CTAS with nested structures
Posted by Adam Gilmore <dr...@gmail.com>.
I submitted a patch for reading all JSON numbers as doubles; however, it'd
probably be nice to extend that to specify a default to read as anything.
Something like ...
alter session set `store.json.read_numbers_as` = 'DECIMAL(5,2)';
would be useful.
On Tue, Feb 24, 2015 at 9:32 AM, Steven Phillips <sp...@maprtech.com>
wrote:
> I don't think this is supported currently.
>
> On Mon, Feb 23, 2015 at 3:29 PM, rahul challapalli <
> challapallirahul@gmail.com> wrote:
>
> > Hi,
> >
> > I want to able to read from a nested json file, cast individual fields,
> and
> > still be able to maintain the nested structure. If this is possible, how
> do
> > I compose my CTAS query?
> >
> > Example :
> >
> > {
> > "map" : {"col1":12.2, "col2":25.5}
> > }
> >
> > The below query generates a flat parquet, but I still want to maintain
> the
> > nestedness
> >
> > select cast(d.map.col1 as decimal(5,2)) , cast(d.map.col2 as double))
> from
> > `data.json`;
> >
> >
> > I am looking for something on the below lines :
> >
> > select cast(m1 as map(col1:decimal, col2:double)) from `data.json`;
> >
> > - Rahul
> >
>
>
>
> --
> Steven Phillips
> Software Engineer
>
> mapr.com
>
Re: Using CTAS with nested structures
Posted by Adam Gilmore <dr...@gmail.com>.
I submitted a patch for reading all JSON numbers as doubles; however, it'd
probably be nice to extend that to specify a default to read as anything.
Something like ...
alter session set `store.json.read_numbers_as` = 'DECIMAL(5,2)';
would be useful.
On Tue, Feb 24, 2015 at 9:32 AM, Steven Phillips <sp...@maprtech.com>
wrote:
> I don't think this is supported currently.
>
> On Mon, Feb 23, 2015 at 3:29 PM, rahul challapalli <
> challapallirahul@gmail.com> wrote:
>
> > Hi,
> >
> > I want to able to read from a nested json file, cast individual fields,
> and
> > still be able to maintain the nested structure. If this is possible, how
> do
> > I compose my CTAS query?
> >
> > Example :
> >
> > {
> > "map" : {"col1":12.2, "col2":25.5}
> > }
> >
> > The below query generates a flat parquet, but I still want to maintain
> the
> > nestedness
> >
> > select cast(d.map.col1 as decimal(5,2)) , cast(d.map.col2 as double))
> from
> > `data.json`;
> >
> >
> > I am looking for something on the below lines :
> >
> > select cast(m1 as map(col1:decimal, col2:double)) from `data.json`;
> >
> > - Rahul
> >
>
>
>
> --
> Steven Phillips
> Software Engineer
>
> mapr.com
>
Re: Using CTAS with nested structures
Posted by Steven Phillips <sp...@maprtech.com>.
I don't think this is supported currently.
On Mon, Feb 23, 2015 at 3:29 PM, rahul challapalli <
challapallirahul@gmail.com> wrote:
> Hi,
>
> I want to able to read from a nested json file, cast individual fields, and
> still be able to maintain the nested structure. If this is possible, how do
> I compose my CTAS query?
>
> Example :
>
> {
> "map" : {"col1":12.2, "col2":25.5}
> }
>
> The below query generates a flat parquet, but I still want to maintain the
> nestedness
>
> select cast(d.map.col1 as decimal(5,2)) , cast(d.map.col2 as double)) from
> `data.json`;
>
>
> I am looking for something on the below lines :
>
> select cast(m1 as map(col1:decimal, col2:double)) from `data.json`;
>
> - Rahul
>
--
Steven Phillips
Software Engineer
mapr.com
Re: Using CTAS with nested structures
Posted by Steven Phillips <sp...@maprtech.com>.
I don't think this is supported currently.
On Mon, Feb 23, 2015 at 3:29 PM, rahul challapalli <
challapallirahul@gmail.com> wrote:
> Hi,
>
> I want to able to read from a nested json file, cast individual fields, and
> still be able to maintain the nested structure. If this is possible, how do
> I compose my CTAS query?
>
> Example :
>
> {
> "map" : {"col1":12.2, "col2":25.5}
> }
>
> The below query generates a flat parquet, but I still want to maintain the
> nestedness
>
> select cast(d.map.col1 as decimal(5,2)) , cast(d.map.col2 as double)) from
> `data.json`;
>
>
> I am looking for something on the below lines :
>
> select cast(m1 as map(col1:decimal, col2:double)) from `data.json`;
>
> - Rahul
>
--
Steven Phillips
Software Engineer
mapr.com