You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Scott Kinney <sc...@stem.com> on 2016/07/01 16:27:39 UTC
array in json with mixed values (int and float)
When running a query on a json file via the api returns an error that i dont see when running the same query in the REPL.
"errorMessage" : "UNSUPPORTED_OPERATION ERROR: In a list of type FLOAT8, encountered a value of type BIGINT. Drill does not support lists of different types.\n\nFile /PowerBladeAvahi.1.telemetry/json/telemetry_flatstore_3_2_prod-telemetry-w2-9_1.log-1437722221.json.gz\nRecord 1\nLine 1\nColumn 502\nField soc\nFragment 0:0\n\n[Error Id: be38e1c4-b1c0-4d55-9ab1-fe4ebdc44a9e on ops-apachedrill:31010]"
I pulled the line out of the file. There is a key 'foo': [ 99, 99.1, 99.8 ].
Is there a way get drill to handle this? Maybe treat all ints as floats?
________________________________
Scott Kinney | DevOps
stem <http://www.stem.com/> | m 510.282.1299
100 Rollins Road, Millbrae, California 94030
This e-mail and/or any attachments contain Stem, Inc. confidential and proprietary information and material for the sole use of the intended recipient(s). Any review, use or distribution that has not been expressly authorized by Stem, Inc. is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. Thank you.
Re: array in json with mixed values (int and float)
Posted by Abdel Hakim Deneche <ad...@maprtech.com>.
This may help:
https://mail-archives.apache.org/mod_mbox/drill-user/201605.mbox/%3CCAKOFcwrdwR2yTZK4a5dGLbOQ0P0yGWtOX6NrMSgRm6NSkk3myA@mail.gmail.com%3E
On Fri, Jul 1, 2016 at 1:26 PM, Parth Chandra <pc...@maprtech.com> wrote:
> If you mean the REST api, then yes, there is no session maintained unless
> impersonation is enabled. In that case the alter session would not have any
> effect.
>
>
>
> On Fri, Jul 1, 2016 at 11:58 AM, Scott Kinney <sc...@stem.com>
> wrote:
>
> > it didn't work when i did an alter session via the api but worked then i
> > did and alter system via the repl. I'm guessing each query via the api
> is a
> > session to alter sessions via the api only last for that one call?
> >
> > Anywho, that did the trick Parth, thank you!
> >
> >
> > ________________________________
> > Scott Kinney | DevOps
> > stem | m 510.282.1299
> > 100 Rollins Road, Millbrae, California 94030
> >
> > This e-mail and/or any attachments contain Stem, Inc. confidential and
> > proprietary information and material for the sole use of the intended
> > recipient(s). Any review, use or distribution that has not been expressly
> > authorized by Stem, Inc. is strictly prohibited. If you are not the
> > intended recipient, please contact the sender and delete all copies.
> Thank
> > you.
> >
> > ________________________________________
> > From: Scott Kinney <sc...@stem.com>
> > Sent: Friday, July 01, 2016 10:51 AM
> > To: user@drill.apache.org
> > Subject: Re: array in json with mixed values (int and float)
> >
> > That looks promising but didn't work.
> >
> >
> > ________________________________
> > Scott Kinney | DevOps
> > stem | m 510.282.1299
> > 100 Rollins Road, Millbrae, California 94030
> >
> > This e-mail and/or any attachments contain Stem, Inc. confidential and
> > proprietary information and material for the sole use of the intended
> > recipient(s). Any review, use or distribution that has not been expressly
> > authorized by Stem, Inc. is strictly prohibited. If you are not the
> > intended recipient, please contact the sender and delete all copies.
> Thank
> > you.
> >
> > ________________________________________
> > From: Parth Chandra <pc...@maprtech.com>
> > Sent: Friday, July 01, 2016 10:43 AM
> > To: user@drill.apache.org
> > Subject: Re: array in json with mixed values (int and float)
> >
> > I haven't tried this myself, but setting
> store.json.read_numbers_as_double
> > to true might help.
> >
> >
> >
> > On Fri, Jul 1, 2016 at 9:27 AM, Scott Kinney <sc...@stem.com>
> > wrote:
> >
> > > When running a query on a json file via the api returns an error that i
> > > dont see when running the same query in the REPL.
> > >
> > > "errorMessage" : "UNSUPPORTED_OPERATION ERROR: In a list of type
> FLOAT8,
> > > encountered a value of type BIGINT. Drill does not support lists of
> > > different types.\n\nFile
> > >
> >
> /PowerBladeAvahi.1.telemetry/json/telemetry_flatstore_3_2_prod-telemetry-w2-9_1.log-1437722221.json.gz\nRecord
> > > 1\nLine 1\nColumn 502\nField soc\nFragment 0:0\n\n[Error Id:
> > > be38e1c4-b1c0-4d55-9ab1-fe4ebdc44a9e on ops-apachedrill:31010]"
> > >
> > >
> > > I pulled the line out of the file. There is a key 'foo': [ 99, 99.1,
> 99.8
> > > ].
> > > Is there a way get drill to handle this? Maybe treat all ints as
> floats?
> > >
> > >
> > > ________________________________
> > > Scott Kinney | DevOps
> > > stem <http://www.stem.com/> | m 510.282.1299
> > > 100 Rollins Road, Millbrae, California 94030
> > >
> > > This e-mail and/or any attachments contain Stem, Inc. confidential and
> > > proprietary information and material for the sole use of the intended
> > > recipient(s). Any review, use or distribution that has not been
> expressly
> > > authorized by Stem, Inc. is strictly prohibited. If you are not the
> > > intended recipient, please contact the sender and delete all copies.
> > Thank
> > > you.
> > >
> >
>
--
Abdelhakim Deneche
Software Engineer
<http://www.mapr.com/>
Now Available - Free Hadoop On-Demand Training
<http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available>
Re: array in json with mixed values (int and float)
Posted by Parth Chandra <pc...@maprtech.com>.
If you mean the REST api, then yes, there is no session maintained unless
impersonation is enabled. In that case the alter session would not have any
effect.
On Fri, Jul 1, 2016 at 11:58 AM, Scott Kinney <sc...@stem.com> wrote:
> it didn't work when i did an alter session via the api but worked then i
> did and alter system via the repl. I'm guessing each query via the api is a
> session to alter sessions via the api only last for that one call?
>
> Anywho, that did the trick Parth, thank you!
>
>
> ________________________________
> Scott Kinney | DevOps
> stem | m 510.282.1299
> 100 Rollins Road, Millbrae, California 94030
>
> This e-mail and/or any attachments contain Stem, Inc. confidential and
> proprietary information and material for the sole use of the intended
> recipient(s). Any review, use or distribution that has not been expressly
> authorized by Stem, Inc. is strictly prohibited. If you are not the
> intended recipient, please contact the sender and delete all copies. Thank
> you.
>
> ________________________________________
> From: Scott Kinney <sc...@stem.com>
> Sent: Friday, July 01, 2016 10:51 AM
> To: user@drill.apache.org
> Subject: Re: array in json with mixed values (int and float)
>
> That looks promising but didn't work.
>
>
> ________________________________
> Scott Kinney | DevOps
> stem | m 510.282.1299
> 100 Rollins Road, Millbrae, California 94030
>
> This e-mail and/or any attachments contain Stem, Inc. confidential and
> proprietary information and material for the sole use of the intended
> recipient(s). Any review, use or distribution that has not been expressly
> authorized by Stem, Inc. is strictly prohibited. If you are not the
> intended recipient, please contact the sender and delete all copies. Thank
> you.
>
> ________________________________________
> From: Parth Chandra <pc...@maprtech.com>
> Sent: Friday, July 01, 2016 10:43 AM
> To: user@drill.apache.org
> Subject: Re: array in json with mixed values (int and float)
>
> I haven't tried this myself, but setting store.json.read_numbers_as_double
> to true might help.
>
>
>
> On Fri, Jul 1, 2016 at 9:27 AM, Scott Kinney <sc...@stem.com>
> wrote:
>
> > When running a query on a json file via the api returns an error that i
> > dont see when running the same query in the REPL.
> >
> > "errorMessage" : "UNSUPPORTED_OPERATION ERROR: In a list of type FLOAT8,
> > encountered a value of type BIGINT. Drill does not support lists of
> > different types.\n\nFile
> >
> /PowerBladeAvahi.1.telemetry/json/telemetry_flatstore_3_2_prod-telemetry-w2-9_1.log-1437722221.json.gz\nRecord
> > 1\nLine 1\nColumn 502\nField soc\nFragment 0:0\n\n[Error Id:
> > be38e1c4-b1c0-4d55-9ab1-fe4ebdc44a9e on ops-apachedrill:31010]"
> >
> >
> > I pulled the line out of the file. There is a key 'foo': [ 99, 99.1, 99.8
> > ].
> > Is there a way get drill to handle this? Maybe treat all ints as floats?
> >
> >
> > ________________________________
> > Scott Kinney | DevOps
> > stem <http://www.stem.com/> | m 510.282.1299
> > 100 Rollins Road, Millbrae, California 94030
> >
> > This e-mail and/or any attachments contain Stem, Inc. confidential and
> > proprietary information and material for the sole use of the intended
> > recipient(s). Any review, use or distribution that has not been expressly
> > authorized by Stem, Inc. is strictly prohibited. If you are not the
> > intended recipient, please contact the sender and delete all copies.
> Thank
> > you.
> >
>
Re: array in json with mixed values (int and float)
Posted by Scott Kinney <sc...@stem.com>.
it didn't work when i did an alter session via the api but worked then i did and alter system via the repl. I'm guessing each query via the api is a session to alter sessions via the api only last for that one call?
Anywho, that did the trick Parth, thank you!
________________________________
Scott Kinney | DevOps
stem | m 510.282.1299
100 Rollins Road, Millbrae, California 94030
This e-mail and/or any attachments contain Stem, Inc. confidential and proprietary information and material for the sole use of the intended recipient(s). Any review, use or distribution that has not been expressly authorized by Stem, Inc. is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. Thank you.
________________________________________
From: Scott Kinney <sc...@stem.com>
Sent: Friday, July 01, 2016 10:51 AM
To: user@drill.apache.org
Subject: Re: array in json with mixed values (int and float)
That looks promising but didn't work.
________________________________
Scott Kinney | DevOps
stem | m 510.282.1299
100 Rollins Road, Millbrae, California 94030
This e-mail and/or any attachments contain Stem, Inc. confidential and proprietary information and material for the sole use of the intended recipient(s). Any review, use or distribution that has not been expressly authorized by Stem, Inc. is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. Thank you.
________________________________________
From: Parth Chandra <pc...@maprtech.com>
Sent: Friday, July 01, 2016 10:43 AM
To: user@drill.apache.org
Subject: Re: array in json with mixed values (int and float)
I haven't tried this myself, but setting store.json.read_numbers_as_double
to true might help.
On Fri, Jul 1, 2016 at 9:27 AM, Scott Kinney <sc...@stem.com> wrote:
> When running a query on a json file via the api returns an error that i
> dont see when running the same query in the REPL.
>
> "errorMessage" : "UNSUPPORTED_OPERATION ERROR: In a list of type FLOAT8,
> encountered a value of type BIGINT. Drill does not support lists of
> different types.\n\nFile
> /PowerBladeAvahi.1.telemetry/json/telemetry_flatstore_3_2_prod-telemetry-w2-9_1.log-1437722221.json.gz\nRecord
> 1\nLine 1\nColumn 502\nField soc\nFragment 0:0\n\n[Error Id:
> be38e1c4-b1c0-4d55-9ab1-fe4ebdc44a9e on ops-apachedrill:31010]"
>
>
> I pulled the line out of the file. There is a key 'foo': [ 99, 99.1, 99.8
> ].
> Is there a way get drill to handle this? Maybe treat all ints as floats?
>
>
> ________________________________
> Scott Kinney | DevOps
> stem <http://www.stem.com/> | m 510.282.1299
> 100 Rollins Road, Millbrae, California 94030
>
> This e-mail and/or any attachments contain Stem, Inc. confidential and
> proprietary information and material for the sole use of the intended
> recipient(s). Any review, use or distribution that has not been expressly
> authorized by Stem, Inc. is strictly prohibited. If you are not the
> intended recipient, please contact the sender and delete all copies. Thank
> you.
>
Re: array in json with mixed values (int and float)
Posted by Scott Kinney <sc...@stem.com>.
That looks promising but didn't work.
________________________________
Scott Kinney | DevOps
stem | m 510.282.1299
100 Rollins Road, Millbrae, California 94030
This e-mail and/or any attachments contain Stem, Inc. confidential and proprietary information and material for the sole use of the intended recipient(s). Any review, use or distribution that has not been expressly authorized by Stem, Inc. is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. Thank you.
________________________________________
From: Parth Chandra <pc...@maprtech.com>
Sent: Friday, July 01, 2016 10:43 AM
To: user@drill.apache.org
Subject: Re: array in json with mixed values (int and float)
I haven't tried this myself, but setting store.json.read_numbers_as_double
to true might help.
On Fri, Jul 1, 2016 at 9:27 AM, Scott Kinney <sc...@stem.com> wrote:
> When running a query on a json file via the api returns an error that i
> dont see when running the same query in the REPL.
>
> "errorMessage" : "UNSUPPORTED_OPERATION ERROR: In a list of type FLOAT8,
> encountered a value of type BIGINT. Drill does not support lists of
> different types.\n\nFile
> /PowerBladeAvahi.1.telemetry/json/telemetry_flatstore_3_2_prod-telemetry-w2-9_1.log-1437722221.json.gz\nRecord
> 1\nLine 1\nColumn 502\nField soc\nFragment 0:0\n\n[Error Id:
> be38e1c4-b1c0-4d55-9ab1-fe4ebdc44a9e on ops-apachedrill:31010]"
>
>
> I pulled the line out of the file. There is a key 'foo': [ 99, 99.1, 99.8
> ].
> Is there a way get drill to handle this? Maybe treat all ints as floats?
>
>
> ________________________________
> Scott Kinney | DevOps
> stem <http://www.stem.com/> | m 510.282.1299
> 100 Rollins Road, Millbrae, California 94030
>
> This e-mail and/or any attachments contain Stem, Inc. confidential and
> proprietary information and material for the sole use of the intended
> recipient(s). Any review, use or distribution that has not been expressly
> authorized by Stem, Inc. is strictly prohibited. If you are not the
> intended recipient, please contact the sender and delete all copies. Thank
> you.
>
Re: array in json with mixed values (int and float)
Posted by Parth Chandra <pc...@maprtech.com>.
I haven't tried this myself, but setting store.json.read_numbers_as_double
to true might help.
On Fri, Jul 1, 2016 at 9:27 AM, Scott Kinney <sc...@stem.com> wrote:
> When running a query on a json file via the api returns an error that i
> dont see when running the same query in the REPL.
>
> "errorMessage" : "UNSUPPORTED_OPERATION ERROR: In a list of type FLOAT8,
> encountered a value of type BIGINT. Drill does not support lists of
> different types.\n\nFile
> /PowerBladeAvahi.1.telemetry/json/telemetry_flatstore_3_2_prod-telemetry-w2-9_1.log-1437722221.json.gz\nRecord
> 1\nLine 1\nColumn 502\nField soc\nFragment 0:0\n\n[Error Id:
> be38e1c4-b1c0-4d55-9ab1-fe4ebdc44a9e on ops-apachedrill:31010]"
>
>
> I pulled the line out of the file. There is a key 'foo': [ 99, 99.1, 99.8
> ].
> Is there a way get drill to handle this? Maybe treat all ints as floats?
>
>
> ________________________________
> Scott Kinney | DevOps
> stem <http://www.stem.com/> | m 510.282.1299
> 100 Rollins Road, Millbrae, California 94030
>
> This e-mail and/or any attachments contain Stem, Inc. confidential and
> proprietary information and material for the sole use of the intended
> recipient(s). Any review, use or distribution that has not been expressly
> authorized by Stem, Inc. is strictly prohibited. If you are not the
> intended recipient, please contact the sender and delete all copies. Thank
> you.
>