You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by John Omernik <jo...@omernik.com> on 2015/10/12 18:45:54 UTC
JSON Data with dot in keyname
I am getting some data that seems well formed, except for there are dots in
some of the key names.
Basically, something like ip.src seems to be making it fail with a "Field
References must be singular names"
I am using the MapR Packaged Drill 1.2.
I can prove this out with
{"hello":"goodbye", "yousayyes":"isayno"}
works fine but
{"hello.yoko":"goodbye", "yousayyes":"isayno"}
Screams loudly that there is a failure.
Is there a way we could handle dots in the key name on JSON?
John
Re: JSON Data with dot in keyname
Posted by Ted Dunning <te...@gmail.com>.
The JIRA is well filled out.
On Mon, Oct 12, 2015 at 10:57 AM, Kristine Hahn <kh...@maprtech.com> wrote:
> lgtm
>
> Kristine Hahn
> Sr. Technical Writer
> 415-497-8107 @krishahn skype:krishahn
>
>
> On Mon, Oct 12, 2015 at 10:45 AM, John Omernik <jo...@omernik.com> wrote:
>
> > I've created https://issues.apache.org/jira/browse/DRILL-3922 for this
> > issue. I put in notes to reproduce, and why it's important (data
> > exploration). Could someone please take a quick look at it to ensure
> that
> > something I may have done in creating the JIRA doesn't affect the chances
> > of it getting looked at? I'm not great at creating JIRAs, and want to
> > ensure I am filling things out correctly so things can get in front of
> the
> > right people. This feature is very important for my test/use cases, and I
> > don't want to inadvertently mess things up by not filling out the JIRA
> > right!
> >
> > John
> >
> >
> > On Mon, Oct 12, 2015 at 12:39 PM, Kristine Hahn <kh...@maprtech.com>
> > wrote:
> >
> > > Yep, it fails on select *:
> > >
> > > select `top.ping`[3].type from
> > >
> > >
> >
> dfs.`/Users/khahn/drill/apache-drill-1.2.0-SNAPSHOT/sample-data/donuts.json`;
> > >
> > > *+-----------------+*
> > >
> > > *| ** EXPR$0 ** |*
> > >
> > > *+-----------------+*
> > >
> > > *| *Powdered Sugar * |*
> > >
> > > *| *null * |*
> > >
> > > *| *null * |*
> > >
> > > *| *null * |*
> > >
> > > *| *null * |*
> > >
> > > *+-----------------+*
> > >
> > > 5 rows selected (0.108 seconds)
> > >
> > > 0: jdbc:drill:zk=local> select * from
> > >
> > >
> >
> dfs.`/Users/khahn/drill/apache-drill-1.2.0-SNAPSHOT/sample-data/donuts.json`;
> > >
> > > *Error: SYSTEM ERROR: UnsupportedOperationException: Field references
> > must
> > > be singular names.*
> > >
> > >
> > > *Fragment 0:0*
> > >
> > >
> > > *[Error Id: a1c586cb-c18f-424f-8758-56adb3f540d2 on 10.250.50.31:31010
> > > <http://10.250.50.31:31010>] (state=,code=0)*
> > >
> > > Kristine Hahn
> > > Sr. Technical Writer
> > > 415-497-8107 @krishahn skype:krishahn
> > >
> > >
> > > On Mon, Oct 12, 2015 at 10:36 AM, John Omernik <jo...@omernik.com>
> wrote:
> > >
> > > > Kristine, if you have that handy, with the first topping to top.ping,
> > can
> > > > you try select * just to validate that it's failing like the data I
> am
> > > > seeing fail? Once I get confirmation, I'll file a JIRA.
> > > >
> > > > On Mon, Oct 12, 2015 at 12:12 PM, Kristine Hahn <kh...@maprtech.com>
> > > > wrote:
> > > >
> > > > > Using the donuts.json
> > > > > <
> > > > >
> > > >
> > >
> >
> https://github.com/apache/drill/blob/master/exec/java-exec/src/test/resources
> > > > > >
> > > > > file, I changed the first topping to top.ping and was able to query
> > > > > successfully using back ticks like this:
> > > > >
> > > > > select `top.ping`[3].type from
> > > > > >
> > > > >
> > > >
> > >
> >
> dfs.`/Users/khahn/drill/apache-drill-1.2.0-SNAPSHOT/sample-data/donuts.json`;
> > > > > > +-----------------+
> > > > > > | EXPR$0 |
> > > > > > +-----------------+
> > > > > > | Powdered Sugar |
> > > > > > | null |
> > > > > > | null |
> > > > > > | null |
> > > > > > | null |
> > > > >
> > > > > *+-----------------+*
> > > > >
> > > > > 5 rows selected (0.108 seconds)
> > > > >
> > > > > Kristine Hahn
> > > > > Sr. Technical Writer
> > > > > 415-497-8107 @krishahn skype:krishahn
> > > > >
> > > > >
> > > > > On Mon, Oct 12, 2015 at 9:45 AM, John Omernik <jo...@omernik.com>
> > > wrote:
> > > > >
> > > > > > I am getting some data that seems well formed, except for there
> are
> > > > dots
> > > > > in
> > > > > > some of the key names.
> > > > > >
> > > > > > Basically, something like ip.src seems to be making it fail with
> a
> > > > "Field
> > > > > > References must be singular names"
> > > > > >
> > > > > > I am using the MapR Packaged Drill 1.2.
> > > > > >
> > > > > >
> > > > > > I can prove this out with
> > > > > >
> > > > > > {"hello":"goodbye", "yousayyes":"isayno"}
> > > > > >
> > > > > > works fine but
> > > > > >
> > > > > > {"hello.yoko":"goodbye", "yousayyes":"isayno"}
> > > > > >
> > > > > > Screams loudly that there is a failure.
> > > > > >
> > > > > > Is there a way we could handle dots in the key name on JSON?
> > > > > >
> > > > > > John
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: JSON Data with dot in keyname
Posted by Kristine Hahn <kh...@maprtech.com>.
lgtm
Kristine Hahn
Sr. Technical Writer
415-497-8107 @krishahn skype:krishahn
On Mon, Oct 12, 2015 at 10:45 AM, John Omernik <jo...@omernik.com> wrote:
> I've created https://issues.apache.org/jira/browse/DRILL-3922 for this
> issue. I put in notes to reproduce, and why it's important (data
> exploration). Could someone please take a quick look at it to ensure that
> something I may have done in creating the JIRA doesn't affect the chances
> of it getting looked at? I'm not great at creating JIRAs, and want to
> ensure I am filling things out correctly so things can get in front of the
> right people. This feature is very important for my test/use cases, and I
> don't want to inadvertently mess things up by not filling out the JIRA
> right!
>
> John
>
>
> On Mon, Oct 12, 2015 at 12:39 PM, Kristine Hahn <kh...@maprtech.com>
> wrote:
>
> > Yep, it fails on select *:
> >
> > select `top.ping`[3].type from
> >
> >
> dfs.`/Users/khahn/drill/apache-drill-1.2.0-SNAPSHOT/sample-data/donuts.json`;
> >
> > *+-----------------+*
> >
> > *| ** EXPR$0 ** |*
> >
> > *+-----------------+*
> >
> > *| *Powdered Sugar * |*
> >
> > *| *null * |*
> >
> > *| *null * |*
> >
> > *| *null * |*
> >
> > *| *null * |*
> >
> > *+-----------------+*
> >
> > 5 rows selected (0.108 seconds)
> >
> > 0: jdbc:drill:zk=local> select * from
> >
> >
> dfs.`/Users/khahn/drill/apache-drill-1.2.0-SNAPSHOT/sample-data/donuts.json`;
> >
> > *Error: SYSTEM ERROR: UnsupportedOperationException: Field references
> must
> > be singular names.*
> >
> >
> > *Fragment 0:0*
> >
> >
> > *[Error Id: a1c586cb-c18f-424f-8758-56adb3f540d2 on 10.250.50.31:31010
> > <http://10.250.50.31:31010>] (state=,code=0)*
> >
> > Kristine Hahn
> > Sr. Technical Writer
> > 415-497-8107 @krishahn skype:krishahn
> >
> >
> > On Mon, Oct 12, 2015 at 10:36 AM, John Omernik <jo...@omernik.com> wrote:
> >
> > > Kristine, if you have that handy, with the first topping to top.ping,
> can
> > > you try select * just to validate that it's failing like the data I am
> > > seeing fail? Once I get confirmation, I'll file a JIRA.
> > >
> > > On Mon, Oct 12, 2015 at 12:12 PM, Kristine Hahn <kh...@maprtech.com>
> > > wrote:
> > >
> > > > Using the donuts.json
> > > > <
> > > >
> > >
> >
> https://github.com/apache/drill/blob/master/exec/java-exec/src/test/resources
> > > > >
> > > > file, I changed the first topping to top.ping and was able to query
> > > > successfully using back ticks like this:
> > > >
> > > > select `top.ping`[3].type from
> > > > >
> > > >
> > >
> >
> dfs.`/Users/khahn/drill/apache-drill-1.2.0-SNAPSHOT/sample-data/donuts.json`;
> > > > > +-----------------+
> > > > > | EXPR$0 |
> > > > > +-----------------+
> > > > > | Powdered Sugar |
> > > > > | null |
> > > > > | null |
> > > > > | null |
> > > > > | null |
> > > >
> > > > *+-----------------+*
> > > >
> > > > 5 rows selected (0.108 seconds)
> > > >
> > > > Kristine Hahn
> > > > Sr. Technical Writer
> > > > 415-497-8107 @krishahn skype:krishahn
> > > >
> > > >
> > > > On Mon, Oct 12, 2015 at 9:45 AM, John Omernik <jo...@omernik.com>
> > wrote:
> > > >
> > > > > I am getting some data that seems well formed, except for there are
> > > dots
> > > > in
> > > > > some of the key names.
> > > > >
> > > > > Basically, something like ip.src seems to be making it fail with a
> > > "Field
> > > > > References must be singular names"
> > > > >
> > > > > I am using the MapR Packaged Drill 1.2.
> > > > >
> > > > >
> > > > > I can prove this out with
> > > > >
> > > > > {"hello":"goodbye", "yousayyes":"isayno"}
> > > > >
> > > > > works fine but
> > > > >
> > > > > {"hello.yoko":"goodbye", "yousayyes":"isayno"}
> > > > >
> > > > > Screams loudly that there is a failure.
> > > > >
> > > > > Is there a way we could handle dots in the key name on JSON?
> > > > >
> > > > > John
> > > > >
> > > >
> > >
> >
>
Re: JSON Data with dot in keyname
Posted by John Omernik <jo...@omernik.com>.
I've created https://issues.apache.org/jira/browse/DRILL-3922 for this
issue. I put in notes to reproduce, and why it's important (data
exploration). Could someone please take a quick look at it to ensure that
something I may have done in creating the JIRA doesn't affect the chances
of it getting looked at? I'm not great at creating JIRAs, and want to
ensure I am filling things out correctly so things can get in front of the
right people. This feature is very important for my test/use cases, and I
don't want to inadvertently mess things up by not filling out the JIRA
right!
John
On Mon, Oct 12, 2015 at 12:39 PM, Kristine Hahn <kh...@maprtech.com> wrote:
> Yep, it fails on select *:
>
> select `top.ping`[3].type from
>
> dfs.`/Users/khahn/drill/apache-drill-1.2.0-SNAPSHOT/sample-data/donuts.json`;
>
> *+-----------------+*
>
> *| ** EXPR$0 ** |*
>
> *+-----------------+*
>
> *| *Powdered Sugar * |*
>
> *| *null * |*
>
> *| *null * |*
>
> *| *null * |*
>
> *| *null * |*
>
> *+-----------------+*
>
> 5 rows selected (0.108 seconds)
>
> 0: jdbc:drill:zk=local> select * from
>
> dfs.`/Users/khahn/drill/apache-drill-1.2.0-SNAPSHOT/sample-data/donuts.json`;
>
> *Error: SYSTEM ERROR: UnsupportedOperationException: Field references must
> be singular names.*
>
>
> *Fragment 0:0*
>
>
> *[Error Id: a1c586cb-c18f-424f-8758-56adb3f540d2 on 10.250.50.31:31010
> <http://10.250.50.31:31010>] (state=,code=0)*
>
> Kristine Hahn
> Sr. Technical Writer
> 415-497-8107 @krishahn skype:krishahn
>
>
> On Mon, Oct 12, 2015 at 10:36 AM, John Omernik <jo...@omernik.com> wrote:
>
> > Kristine, if you have that handy, with the first topping to top.ping, can
> > you try select * just to validate that it's failing like the data I am
> > seeing fail? Once I get confirmation, I'll file a JIRA.
> >
> > On Mon, Oct 12, 2015 at 12:12 PM, Kristine Hahn <kh...@maprtech.com>
> > wrote:
> >
> > > Using the donuts.json
> > > <
> > >
> >
> https://github.com/apache/drill/blob/master/exec/java-exec/src/test/resources
> > > >
> > > file, I changed the first topping to top.ping and was able to query
> > > successfully using back ticks like this:
> > >
> > > select `top.ping`[3].type from
> > > >
> > >
> >
> dfs.`/Users/khahn/drill/apache-drill-1.2.0-SNAPSHOT/sample-data/donuts.json`;
> > > > +-----------------+
> > > > | EXPR$0 |
> > > > +-----------------+
> > > > | Powdered Sugar |
> > > > | null |
> > > > | null |
> > > > | null |
> > > > | null |
> > >
> > > *+-----------------+*
> > >
> > > 5 rows selected (0.108 seconds)
> > >
> > > Kristine Hahn
> > > Sr. Technical Writer
> > > 415-497-8107 @krishahn skype:krishahn
> > >
> > >
> > > On Mon, Oct 12, 2015 at 9:45 AM, John Omernik <jo...@omernik.com>
> wrote:
> > >
> > > > I am getting some data that seems well formed, except for there are
> > dots
> > > in
> > > > some of the key names.
> > > >
> > > > Basically, something like ip.src seems to be making it fail with a
> > "Field
> > > > References must be singular names"
> > > >
> > > > I am using the MapR Packaged Drill 1.2.
> > > >
> > > >
> > > > I can prove this out with
> > > >
> > > > {"hello":"goodbye", "yousayyes":"isayno"}
> > > >
> > > > works fine but
> > > >
> > > > {"hello.yoko":"goodbye", "yousayyes":"isayno"}
> > > >
> > > > Screams loudly that there is a failure.
> > > >
> > > > Is there a way we could handle dots in the key name on JSON?
> > > >
> > > > John
> > > >
> > >
> >
>
Re: JSON Data with dot in keyname
Posted by Kristine Hahn <kh...@maprtech.com>.
Yep, it fails on select *:
select `top.ping`[3].type from
dfs.`/Users/khahn/drill/apache-drill-1.2.0-SNAPSHOT/sample-data/donuts.json`;
*+-----------------+*
*| ** EXPR$0 ** |*
*+-----------------+*
*| *Powdered Sugar * |*
*| *null * |*
*| *null * |*
*| *null * |*
*| *null * |*
*+-----------------+*
5 rows selected (0.108 seconds)
0: jdbc:drill:zk=local> select * from
dfs.`/Users/khahn/drill/apache-drill-1.2.0-SNAPSHOT/sample-data/donuts.json`;
*Error: SYSTEM ERROR: UnsupportedOperationException: Field references must
be singular names.*
*Fragment 0:0*
*[Error Id: a1c586cb-c18f-424f-8758-56adb3f540d2 on 10.250.50.31:31010
<http://10.250.50.31:31010>] (state=,code=0)*
Kristine Hahn
Sr. Technical Writer
415-497-8107 @krishahn skype:krishahn
On Mon, Oct 12, 2015 at 10:36 AM, John Omernik <jo...@omernik.com> wrote:
> Kristine, if you have that handy, with the first topping to top.ping, can
> you try select * just to validate that it's failing like the data I am
> seeing fail? Once I get confirmation, I'll file a JIRA.
>
> On Mon, Oct 12, 2015 at 12:12 PM, Kristine Hahn <kh...@maprtech.com>
> wrote:
>
> > Using the donuts.json
> > <
> >
> https://github.com/apache/drill/blob/master/exec/java-exec/src/test/resources
> > >
> > file, I changed the first topping to top.ping and was able to query
> > successfully using back ticks like this:
> >
> > select `top.ping`[3].type from
> > >
> >
> dfs.`/Users/khahn/drill/apache-drill-1.2.0-SNAPSHOT/sample-data/donuts.json`;
> > > +-----------------+
> > > | EXPR$0 |
> > > +-----------------+
> > > | Powdered Sugar |
> > > | null |
> > > | null |
> > > | null |
> > > | null |
> >
> > *+-----------------+*
> >
> > 5 rows selected (0.108 seconds)
> >
> > Kristine Hahn
> > Sr. Technical Writer
> > 415-497-8107 @krishahn skype:krishahn
> >
> >
> > On Mon, Oct 12, 2015 at 9:45 AM, John Omernik <jo...@omernik.com> wrote:
> >
> > > I am getting some data that seems well formed, except for there are
> dots
> > in
> > > some of the key names.
> > >
> > > Basically, something like ip.src seems to be making it fail with a
> "Field
> > > References must be singular names"
> > >
> > > I am using the MapR Packaged Drill 1.2.
> > >
> > >
> > > I can prove this out with
> > >
> > > {"hello":"goodbye", "yousayyes":"isayno"}
> > >
> > > works fine but
> > >
> > > {"hello.yoko":"goodbye", "yousayyes":"isayno"}
> > >
> > > Screams loudly that there is a failure.
> > >
> > > Is there a way we could handle dots in the key name on JSON?
> > >
> > > John
> > >
> >
>
Re: JSON Data with dot in keyname
Posted by John Omernik <jo...@omernik.com>.
Kristine, if you have that handy, with the first topping to top.ping, can
you try select * just to validate that it's failing like the data I am
seeing fail? Once I get confirmation, I'll file a JIRA.
On Mon, Oct 12, 2015 at 12:12 PM, Kristine Hahn <kh...@maprtech.com> wrote:
> Using the donuts.json
> <
> https://github.com/apache/drill/blob/master/exec/java-exec/src/test/resources
> >
> file, I changed the first topping to top.ping and was able to query
> successfully using back ticks like this:
>
> select `top.ping`[3].type from
> >
> dfs.`/Users/khahn/drill/apache-drill-1.2.0-SNAPSHOT/sample-data/donuts.json`;
> > +-----------------+
> > | EXPR$0 |
> > +-----------------+
> > | Powdered Sugar |
> > | null |
> > | null |
> > | null |
> > | null |
>
> *+-----------------+*
>
> 5 rows selected (0.108 seconds)
>
> Kristine Hahn
> Sr. Technical Writer
> 415-497-8107 @krishahn skype:krishahn
>
>
> On Mon, Oct 12, 2015 at 9:45 AM, John Omernik <jo...@omernik.com> wrote:
>
> > I am getting some data that seems well formed, except for there are dots
> in
> > some of the key names.
> >
> > Basically, something like ip.src seems to be making it fail with a "Field
> > References must be singular names"
> >
> > I am using the MapR Packaged Drill 1.2.
> >
> >
> > I can prove this out with
> >
> > {"hello":"goodbye", "yousayyes":"isayno"}
> >
> > works fine but
> >
> > {"hello.yoko":"goodbye", "yousayyes":"isayno"}
> >
> > Screams loudly that there is a failure.
> >
> > Is there a way we could handle dots in the key name on JSON?
> >
> > John
> >
>
Re: JSON Data with dot in keyname
Posted by Kristine Hahn <kh...@maprtech.com>.
Using the donuts.json
<https://github.com/apache/drill/blob/master/exec/java-exec/src/test/resources>
file, I changed the first topping to top.ping and was able to query
successfully using back ticks like this:
select `top.ping`[3].type from
> dfs.`/Users/khahn/drill/apache-drill-1.2.0-SNAPSHOT/sample-data/donuts.json`;
> +-----------------+
> | EXPR$0 |
> +-----------------+
> | Powdered Sugar |
> | null |
> | null |
> | null |
> | null |
*+-----------------+*
5 rows selected (0.108 seconds)
Kristine Hahn
Sr. Technical Writer
415-497-8107 @krishahn skype:krishahn
On Mon, Oct 12, 2015 at 9:45 AM, John Omernik <jo...@omernik.com> wrote:
> I am getting some data that seems well formed, except for there are dots in
> some of the key names.
>
> Basically, something like ip.src seems to be making it fail with a "Field
> References must be singular names"
>
> I am using the MapR Packaged Drill 1.2.
>
>
> I can prove this out with
>
> {"hello":"goodbye", "yousayyes":"isayno"}
>
> works fine but
>
> {"hello.yoko":"goodbye", "yousayyes":"isayno"}
>
> Screams loudly that there is a failure.
>
> Is there a way we could handle dots in the key name on JSON?
>
> John
>