You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by John Omernik <jo...@omernik.com> on 2015/09/21 21:11:24 UTC
JSON Using "keys" as values
I have some poorly developed json where the developer used data for key
names
{"created":"2015-12-01", "ZYS":"BLAH"}
{"created":"2015-12-01", "ZYX":"BLAH"}
{"created":"2015-12-01", "ABC":"BLAH"}
{"created":"2015-12-01", "ADS":"BLAH"}
I'd like to somehow map the key name to a value and give it a generic name
select `created`, somemagic() as value1 from table
Not sure how this would work, or if it's possible, or how I'd even
reference that, but thought I would ask.
John
JSON Using "keys" as values
Posted by Christopher Matta <cm...@mapr.com>.
This has come up a few times in the past, maybe this is an opportunity to
figure out the best way to handle poorly created JSON records at scale?
Without having to transform it?
Chris Matta
cmatta@mapr.com <javascript:_e(%7B%7D,'cvml','cmatta@mapr.com');>
215-701-3146
On Mon, Sep 21, 2015 at 3:53 PM, John Omernik <john@omernik.com
<javascript:_e(%7B%7D,'cvml','john@omernik.com');>> wrote:
> Sorry about that, premature send. Here are some records. As you can see
> the myvalue1-3 is in the top level of the record, ideally I'd run thte
> kvgen on the myvalue records, but I have no way to address those. I tried
> kvgen() on * for and that failed. Not sure how to address this in json,
> yes, I know it's poorly formatted, but it's what I have been given.
>
>
>
>
>
> { "_id" : "127.0.0.1", "created_on" : "2014-02-18 14:52:23", "myvalue1" : {
> "source" : "somestuff", "context" : "Context here", "last_seen" :
> "2014-02-11 00:00:00", "refreshed" : "2014-03-12 18:14:23" } }
> { "_id" : "127.0.0.2", "created_on" : "2014-02-18 14:52:08", "myvalue2" : {
> "source" : "otherstuff", "context" : "Special context", "last_seen" :
> "2014-02-26 18:14:05", "refreshed" : "2014-02-26 18:14:05" } }
> { "_id" : "127.0.0.3", "created_on" : "2014-04-25 00:08:17", "myvalue3" : {
> "source" : "oops", "context" : "Other Context, "last_seen" : "2014-04-25
> 05:32:08", "refreshed" : "2014-04-25 05:32:08" } }
>
> On Mon, Sep 21, 2015 at 2:52 PM, John Omernik <john@omernik.com
> <javascript:_e(%7B%7D,'cvml','john@omernik.com');>> wrote:
>
> > The challenge I have is the data was poorly formatted, here's some
> records
> >
> >
> >
> >
> >
> > On Mon, Sep 21, 2015 at 2:14 PM, Jim Scott <jscott@maprtech.com
> <javascript:_e(%7B%7D,'cvml','jscott@maprtech.com');>> wrote:
> >
> >> I do believe KVGEN will meet your needs:
> >> https://drill.apache.org/docs/kvgen/
> >>
> >> On Mon, Sep 21, 2015 at 2:11 PM, John Omernik <john@omernik.com
> <javascript:_e(%7B%7D,'cvml','john@omernik.com');>> wrote:
> >>
> >> > I have some poorly developed json where the developer used data for
> key
> >> > names
> >> >
> >> > {"created":"2015-12-01", "ZYS":"BLAH"}
> >> > {"created":"2015-12-01", "ZYX":"BLAH"}
> >> > {"created":"2015-12-01", "ABC":"BLAH"}
> >> > {"created":"2015-12-01", "ADS":"BLAH"}
> >> >
> >> > I'd like to somehow map the key name to a value and give it a generic
> >> name
> >> >
> >> > select `created`, somemagic() as value1 from table
> >> >
> >> > Not sure how this would work, or if it's possible, or how I'd even
> >> > reference that, but thought I would ask.
> >> >
> >> > John
> >> >
> >>
> >>
> >>
> >> --
> >> *Jim Scott*
> >> Director, Enterprise Strategy & Architecture
> >> +1 (347) 746-9281
> >> @kingmesal <https://twitter.com/kingmesal>
> >>
> >> <http://www.mapr.com/>
> >> [image: MapR Technologies] <http://www.mapr.com>
> >>
> >> Now Available - Free Hadoop On-Demand Training
> >> <
> >>
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> >> >
> >>
> >
> >
>
--
Chris Matta
cmatta@mapr.com
215-701-3146
Re: JSON Using "keys" as values
Posted by Ted Dunning <te...@gmail.com>.
But that doesn't work either. Dreaded type change error.
On Mon, Sep 21, 2015 at 5:17 PM, Ted Dunning <te...@gmail.com> wrote:
>
> One hack is to wrap each record with {"data": ... }
>
> That gives you something to grab onto.
>
> On Mon, Sep 21, 2015 at 12:53 PM, John Omernik <jo...@omernik.com> wrote:
>
>> Sorry about that, premature send. Here are some records. As you can see
>> the myvalue1-3 is in the top level of the record, ideally I'd run thte
>> kvgen on the myvalue records, but I have no way to address those. I tried
>> kvgen() on * for and that failed. Not sure how to address this in json,
>> yes, I know it's poorly formatted, but it's what I have been given.
>>
>>
>>
>>
>>
>> { "_id" : "127.0.0.1", "created_on" : "2014-02-18 14:52:23", "myvalue1" :
>> {
>> "source" : "somestuff", "context" : "Context here", "last_seen" :
>> "2014-02-11 00:00:00", "refreshed" : "2014-03-12 18:14:23" } }
>> { "_id" : "127.0.0.2", "created_on" : "2014-02-18 14:52:08", "myvalue2" :
>> {
>> "source" : "otherstuff", "context" : "Special context", "last_seen" :
>> "2014-02-26 18:14:05", "refreshed" : "2014-02-26 18:14:05" } }
>> { "_id" : "127.0.0.3", "created_on" : "2014-04-25 00:08:17", "myvalue3" :
>> {
>> "source" : "oops", "context" : "Other Context, "last_seen" : "2014-04-25
>> 05:32:08", "refreshed" : "2014-04-25 05:32:08" } }
>>
>> On Mon, Sep 21, 2015 at 2:52 PM, John Omernik <jo...@omernik.com> wrote:
>>
>> > The challenge I have is the data was poorly formatted, here's some
>> records
>> >
>> >
>> >
>> >
>> >
>> > On Mon, Sep 21, 2015 at 2:14 PM, Jim Scott <js...@maprtech.com> wrote:
>> >
>> >> I do believe KVGEN will meet your needs:
>> >> https://drill.apache.org/docs/kvgen/
>> >>
>> >> On Mon, Sep 21, 2015 at 2:11 PM, John Omernik <jo...@omernik.com>
>> wrote:
>> >>
>> >> > I have some poorly developed json where the developer used data for
>> key
>> >> > names
>> >> >
>> >> > {"created":"2015-12-01", "ZYS":"BLAH"}
>> >> > {"created":"2015-12-01", "ZYX":"BLAH"}
>> >> > {"created":"2015-12-01", "ABC":"BLAH"}
>> >> > {"created":"2015-12-01", "ADS":"BLAH"}
>> >> >
>> >> > I'd like to somehow map the key name to a value and give it a generic
>> >> name
>> >> >
>> >> > select `created`, somemagic() as value1 from table
>> >> >
>> >> > Not sure how this would work, or if it's possible, or how I'd even
>> >> > reference that, but thought I would ask.
>> >> >
>> >> > John
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> *Jim Scott*
>> >> Director, Enterprise Strategy & Architecture
>> >> +1 (347) 746-9281
>> >> @kingmesal <https://twitter.com/kingmesal>
>> >>
>> >> <http://www.mapr.com/>
>> >> [image: MapR Technologies] <http://www.mapr.com>
>> >>
>> >> Now Available - Free Hadoop On-Demand Training
>> >> <
>> >>
>> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
>> >> >
>> >>
>> >
>> >
>>
>
>
Re: JSON Using "keys" as values
Posted by Ted Dunning <te...@gmail.com>.
One hack is to wrap each record with {"data": ... }
That gives you something to grab onto.
On Mon, Sep 21, 2015 at 12:53 PM, John Omernik <jo...@omernik.com> wrote:
> Sorry about that, premature send. Here are some records. As you can see
> the myvalue1-3 is in the top level of the record, ideally I'd run thte
> kvgen on the myvalue records, but I have no way to address those. I tried
> kvgen() on * for and that failed. Not sure how to address this in json,
> yes, I know it's poorly formatted, but it's what I have been given.
>
>
>
>
>
> { "_id" : "127.0.0.1", "created_on" : "2014-02-18 14:52:23", "myvalue1" : {
> "source" : "somestuff", "context" : "Context here", "last_seen" :
> "2014-02-11 00:00:00", "refreshed" : "2014-03-12 18:14:23" } }
> { "_id" : "127.0.0.2", "created_on" : "2014-02-18 14:52:08", "myvalue2" : {
> "source" : "otherstuff", "context" : "Special context", "last_seen" :
> "2014-02-26 18:14:05", "refreshed" : "2014-02-26 18:14:05" } }
> { "_id" : "127.0.0.3", "created_on" : "2014-04-25 00:08:17", "myvalue3" : {
> "source" : "oops", "context" : "Other Context, "last_seen" : "2014-04-25
> 05:32:08", "refreshed" : "2014-04-25 05:32:08" } }
>
> On Mon, Sep 21, 2015 at 2:52 PM, John Omernik <jo...@omernik.com> wrote:
>
> > The challenge I have is the data was poorly formatted, here's some
> records
> >
> >
> >
> >
> >
> > On Mon, Sep 21, 2015 at 2:14 PM, Jim Scott <js...@maprtech.com> wrote:
> >
> >> I do believe KVGEN will meet your needs:
> >> https://drill.apache.org/docs/kvgen/
> >>
> >> On Mon, Sep 21, 2015 at 2:11 PM, John Omernik <jo...@omernik.com> wrote:
> >>
> >> > I have some poorly developed json where the developer used data for
> key
> >> > names
> >> >
> >> > {"created":"2015-12-01", "ZYS":"BLAH"}
> >> > {"created":"2015-12-01", "ZYX":"BLAH"}
> >> > {"created":"2015-12-01", "ABC":"BLAH"}
> >> > {"created":"2015-12-01", "ADS":"BLAH"}
> >> >
> >> > I'd like to somehow map the key name to a value and give it a generic
> >> name
> >> >
> >> > select `created`, somemagic() as value1 from table
> >> >
> >> > Not sure how this would work, or if it's possible, or how I'd even
> >> > reference that, but thought I would ask.
> >> >
> >> > John
> >> >
> >>
> >>
> >>
> >> --
> >> *Jim Scott*
> >> Director, Enterprise Strategy & Architecture
> >> +1 (347) 746-9281
> >> @kingmesal <https://twitter.com/kingmesal>
> >>
> >> <http://www.mapr.com/>
> >> [image: MapR Technologies] <http://www.mapr.com>
> >>
> >> Now Available - Free Hadoop On-Demand Training
> >> <
> >>
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> >> >
> >>
> >
> >
>
Re: JSON Using "keys" as values
Posted by John Omernik <jo...@omernik.com>.
I hacked around this using a spark. Basically I had spark read the files,
use the json module in python, read in, adjusted the dictionaries to be an
array (and added the name) and then had spark output to parquet. Made a
drill view to clean up some things. (when I created the array in a
"myvalues" key, instead of myvalues:[{dic1}, {dict2}] etc
it did
myvalues:{bag:[{dic1},{dic2}]}
which was weird, spark thing maybe? I don't have the string "bag" anywhere
in my code. So I am not sure how it got there. But it's working!
I made a view on that data to clean it up (removed the bag etc) and it
works well!
John
On Tue, Sep 22, 2015 at 7:35 AM, John Omernik <jo...@omernik.com> wrote:
> Tomer -
>
> I wrote a hacky Python script to get it all out and came up with 263
> values..
>
> Chris - This is an interesting problem. If there were some semantics in
> the json plugin to handle this, it would be helpful. The challenge as I was
> thinking through it is mongo isn't consistent in how it outputs the data,
> in the three I included in the thread its _id, created_on, and the dynamic
> myvalues, however, looking through the data, there are other records where
> created_on is the last one, so even parsing it as an array, and using the
> array value to reference it doesn't help. This is a challenge!
>
>
>
> On Mon, Sep 21, 2015 at 7:52 PM, Tomer Shiran <ts...@dremio.com> wrote:
>
>> How many different myvalueX are there in your dataset? (In the example
>> below you have 3.) is it a small, known set, or could it be anything?
>>
>>
>>
>> > On Sep 21, 2015, at 12:53 PM, John Omernik <jo...@omernik.com> wrote:
>> >
>> > Sorry about that, premature send. Here are some records. As you can
>> see
>> > the myvalue1-3 is in the top level of the record, ideally I'd run thte
>> > kvgen on the myvalue records, but I have no way to address those. I
>> tried
>> > kvgen() on * for and that failed. Not sure how to address this in json,
>> > yes, I know it's poorly formatted, but it's what I have been given.
>> >
>> >
>> >
>> >
>> >
>> > { "_id" : "127.0.0.1", "created_on" : "2014-02-18 14:52:23", "myvalue1"
>> : {
>> > "source" : "somestuff", "context" : "Context here", "last_seen" :
>> > "2014-02-11 00:00:00", "refreshed" : "2014-03-12 18:14:23" } }
>> > { "_id" : "127.0.0.2", "created_on" : "2014-02-18 14:52:08", "myvalue2"
>> : {
>> > "source" : "otherstuff", "context" : "Special context", "last_seen" :
>> > "2014-02-26 18:14:05", "refreshed" : "2014-02-26 18:14:05" } }
>> > { "_id" : "127.0.0.3", "created_on" : "2014-04-25 00:08:17", "myvalue3"
>> : {
>> > "source" : "oops", "context" : "Other Context, "last_seen" : "2014-04-25
>> > 05:32:08", "refreshed" : "2014-04-25 05:32:08" } }
>> >
>> >> On Mon, Sep 21, 2015 at 2:52 PM, John Omernik <jo...@omernik.com>
>> wrote:
>> >>
>> >> The challenge I have is the data was poorly formatted, here's some
>> records
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>> On Mon, Sep 21, 2015 at 2:14 PM, Jim Scott <js...@maprtech.com>
>> wrote:
>> >>>
>> >>> I do believe KVGEN will meet your needs:
>> >>> https://drill.apache.org/docs/kvgen/
>> >>>
>> >>>> On Mon, Sep 21, 2015 at 2:11 PM, John Omernik <jo...@omernik.com>
>> wrote:
>> >>>>
>> >>>> I have some poorly developed json where the developer used data for
>> key
>> >>>> names
>> >>>>
>> >>>> {"created":"2015-12-01", "ZYS":"BLAH"}
>> >>>> {"created":"2015-12-01", "ZYX":"BLAH"}
>> >>>> {"created":"2015-12-01", "ABC":"BLAH"}
>> >>>> {"created":"2015-12-01", "ADS":"BLAH"}
>> >>>>
>> >>>> I'd like to somehow map the key name to a value and give it a generic
>> >>> name
>> >>>>
>> >>>> select `created`, somemagic() as value1 from table
>> >>>>
>> >>>> Not sure how this would work, or if it's possible, or how I'd even
>> >>>> reference that, but thought I would ask.
>> >>>>
>> >>>> John
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> *Jim Scott*
>> >>> Director, Enterprise Strategy & Architecture
>> >>> +1 (347) 746-9281
>> >>> @kingmesal <https://twitter.com/kingmesal>
>> >>>
>> >>> <http://www.mapr.com/>
>> >>> [image: MapR Technologies] <http://www.mapr.com>
>> >>>
>> >>> Now Available - Free Hadoop On-Demand Training
>> >>> <
>> >>>
>> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
>> >>
>> >>
>>
>
>
Re: JSON Using "keys" as values
Posted by John Omernik <jo...@omernik.com>.
Tomer -
I wrote a hacky Python script to get it all out and came up with 263
values..
Chris - This is an interesting problem. If there were some semantics in
the json plugin to handle this, it would be helpful. The challenge as I was
thinking through it is mongo isn't consistent in how it outputs the data,
in the three I included in the thread its _id, created_on, and the dynamic
myvalues, however, looking through the data, there are other records where
created_on is the last one, so even parsing it as an array, and using the
array value to reference it doesn't help. This is a challenge!
On Mon, Sep 21, 2015 at 7:52 PM, Tomer Shiran <ts...@dremio.com> wrote:
> How many different myvalueX are there in your dataset? (In the example
> below you have 3.) is it a small, known set, or could it be anything?
>
>
>
> > On Sep 21, 2015, at 12:53 PM, John Omernik <jo...@omernik.com> wrote:
> >
> > Sorry about that, premature send. Here are some records. As you can see
> > the myvalue1-3 is in the top level of the record, ideally I'd run thte
> > kvgen on the myvalue records, but I have no way to address those. I
> tried
> > kvgen() on * for and that failed. Not sure how to address this in json,
> > yes, I know it's poorly formatted, but it's what I have been given.
> >
> >
> >
> >
> >
> > { "_id" : "127.0.0.1", "created_on" : "2014-02-18 14:52:23", "myvalue1"
> : {
> > "source" : "somestuff", "context" : "Context here", "last_seen" :
> > "2014-02-11 00:00:00", "refreshed" : "2014-03-12 18:14:23" } }
> > { "_id" : "127.0.0.2", "created_on" : "2014-02-18 14:52:08", "myvalue2"
> : {
> > "source" : "otherstuff", "context" : "Special context", "last_seen" :
> > "2014-02-26 18:14:05", "refreshed" : "2014-02-26 18:14:05" } }
> > { "_id" : "127.0.0.3", "created_on" : "2014-04-25 00:08:17", "myvalue3"
> : {
> > "source" : "oops", "context" : "Other Context, "last_seen" : "2014-04-25
> > 05:32:08", "refreshed" : "2014-04-25 05:32:08" } }
> >
> >> On Mon, Sep 21, 2015 at 2:52 PM, John Omernik <jo...@omernik.com> wrote:
> >>
> >> The challenge I have is the data was poorly formatted, here's some
> records
> >>
> >>
> >>
> >>
> >>
> >>> On Mon, Sep 21, 2015 at 2:14 PM, Jim Scott <js...@maprtech.com>
> wrote:
> >>>
> >>> I do believe KVGEN will meet your needs:
> >>> https://drill.apache.org/docs/kvgen/
> >>>
> >>>> On Mon, Sep 21, 2015 at 2:11 PM, John Omernik <jo...@omernik.com>
> wrote:
> >>>>
> >>>> I have some poorly developed json where the developer used data for
> key
> >>>> names
> >>>>
> >>>> {"created":"2015-12-01", "ZYS":"BLAH"}
> >>>> {"created":"2015-12-01", "ZYX":"BLAH"}
> >>>> {"created":"2015-12-01", "ABC":"BLAH"}
> >>>> {"created":"2015-12-01", "ADS":"BLAH"}
> >>>>
> >>>> I'd like to somehow map the key name to a value and give it a generic
> >>> name
> >>>>
> >>>> select `created`, somemagic() as value1 from table
> >>>>
> >>>> Not sure how this would work, or if it's possible, or how I'd even
> >>>> reference that, but thought I would ask.
> >>>>
> >>>> John
> >>>
> >>>
> >>>
> >>> --
> >>> *Jim Scott*
> >>> Director, Enterprise Strategy & Architecture
> >>> +1 (347) 746-9281
> >>> @kingmesal <https://twitter.com/kingmesal>
> >>>
> >>> <http://www.mapr.com/>
> >>> [image: MapR Technologies] <http://www.mapr.com>
> >>>
> >>> Now Available - Free Hadoop On-Demand Training
> >>> <
> >>>
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> >>
> >>
>
Re: JSON Using "keys" as values
Posted by Tomer Shiran <ts...@dremio.com>.
How many different myvalueX are there in your dataset? (In the example below you have 3.) is it a small, known set, or could it be anything?
> On Sep 21, 2015, at 12:53 PM, John Omernik <jo...@omernik.com> wrote:
>
> Sorry about that, premature send. Here are some records. As you can see
> the myvalue1-3 is in the top level of the record, ideally I'd run thte
> kvgen on the myvalue records, but I have no way to address those. I tried
> kvgen() on * for and that failed. Not sure how to address this in json,
> yes, I know it's poorly formatted, but it's what I have been given.
>
>
>
>
>
> { "_id" : "127.0.0.1", "created_on" : "2014-02-18 14:52:23", "myvalue1" : {
> "source" : "somestuff", "context" : "Context here", "last_seen" :
> "2014-02-11 00:00:00", "refreshed" : "2014-03-12 18:14:23" } }
> { "_id" : "127.0.0.2", "created_on" : "2014-02-18 14:52:08", "myvalue2" : {
> "source" : "otherstuff", "context" : "Special context", "last_seen" :
> "2014-02-26 18:14:05", "refreshed" : "2014-02-26 18:14:05" } }
> { "_id" : "127.0.0.3", "created_on" : "2014-04-25 00:08:17", "myvalue3" : {
> "source" : "oops", "context" : "Other Context, "last_seen" : "2014-04-25
> 05:32:08", "refreshed" : "2014-04-25 05:32:08" } }
>
>> On Mon, Sep 21, 2015 at 2:52 PM, John Omernik <jo...@omernik.com> wrote:
>>
>> The challenge I have is the data was poorly formatted, here's some records
>>
>>
>>
>>
>>
>>> On Mon, Sep 21, 2015 at 2:14 PM, Jim Scott <js...@maprtech.com> wrote:
>>>
>>> I do believe KVGEN will meet your needs:
>>> https://drill.apache.org/docs/kvgen/
>>>
>>>> On Mon, Sep 21, 2015 at 2:11 PM, John Omernik <jo...@omernik.com> wrote:
>>>>
>>>> I have some poorly developed json where the developer used data for key
>>>> names
>>>>
>>>> {"created":"2015-12-01", "ZYS":"BLAH"}
>>>> {"created":"2015-12-01", "ZYX":"BLAH"}
>>>> {"created":"2015-12-01", "ABC":"BLAH"}
>>>> {"created":"2015-12-01", "ADS":"BLAH"}
>>>>
>>>> I'd like to somehow map the key name to a value and give it a generic
>>> name
>>>>
>>>> select `created`, somemagic() as value1 from table
>>>>
>>>> Not sure how this would work, or if it's possible, or how I'd even
>>>> reference that, but thought I would ask.
>>>>
>>>> John
>>>
>>>
>>>
>>> --
>>> *Jim Scott*
>>> Director, Enterprise Strategy & Architecture
>>> +1 (347) 746-9281
>>> @kingmesal <https://twitter.com/kingmesal>
>>>
>>> <http://www.mapr.com/>
>>> [image: MapR Technologies] <http://www.mapr.com>
>>>
>>> Now Available - Free Hadoop On-Demand Training
>>> <
>>> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
>>
>>
Re: JSON Using "keys" as values
Posted by John Omernik <jo...@omernik.com>.
Sorry about that, premature send. Here are some records. As you can see
the myvalue1-3 is in the top level of the record, ideally I'd run thte
kvgen on the myvalue records, but I have no way to address those. I tried
kvgen() on * for and that failed. Not sure how to address this in json,
yes, I know it's poorly formatted, but it's what I have been given.
{ "_id" : "127.0.0.1", "created_on" : "2014-02-18 14:52:23", "myvalue1" : {
"source" : "somestuff", "context" : "Context here", "last_seen" :
"2014-02-11 00:00:00", "refreshed" : "2014-03-12 18:14:23" } }
{ "_id" : "127.0.0.2", "created_on" : "2014-02-18 14:52:08", "myvalue2" : {
"source" : "otherstuff", "context" : "Special context", "last_seen" :
"2014-02-26 18:14:05", "refreshed" : "2014-02-26 18:14:05" } }
{ "_id" : "127.0.0.3", "created_on" : "2014-04-25 00:08:17", "myvalue3" : {
"source" : "oops", "context" : "Other Context, "last_seen" : "2014-04-25
05:32:08", "refreshed" : "2014-04-25 05:32:08" } }
On Mon, Sep 21, 2015 at 2:52 PM, John Omernik <jo...@omernik.com> wrote:
> The challenge I have is the data was poorly formatted, here's some records
>
>
>
>
>
> On Mon, Sep 21, 2015 at 2:14 PM, Jim Scott <js...@maprtech.com> wrote:
>
>> I do believe KVGEN will meet your needs:
>> https://drill.apache.org/docs/kvgen/
>>
>> On Mon, Sep 21, 2015 at 2:11 PM, John Omernik <jo...@omernik.com> wrote:
>>
>> > I have some poorly developed json where the developer used data for key
>> > names
>> >
>> > {"created":"2015-12-01", "ZYS":"BLAH"}
>> > {"created":"2015-12-01", "ZYX":"BLAH"}
>> > {"created":"2015-12-01", "ABC":"BLAH"}
>> > {"created":"2015-12-01", "ADS":"BLAH"}
>> >
>> > I'd like to somehow map the key name to a value and give it a generic
>> name
>> >
>> > select `created`, somemagic() as value1 from table
>> >
>> > Not sure how this would work, or if it's possible, or how I'd even
>> > reference that, but thought I would ask.
>> >
>> > John
>> >
>>
>>
>>
>> --
>> *Jim Scott*
>> Director, Enterprise Strategy & Architecture
>> +1 (347) 746-9281
>> @kingmesal <https://twitter.com/kingmesal>
>>
>> <http://www.mapr.com/>
>> [image: MapR Technologies] <http://www.mapr.com>
>>
>> Now Available - Free Hadoop On-Demand Training
>> <
>> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
>> >
>>
>
>
Re: JSON Using "keys" as values
Posted by John Omernik <jo...@omernik.com>.
The challenge I have is the data was poorly formatted, here's some records
On Mon, Sep 21, 2015 at 2:14 PM, Jim Scott <js...@maprtech.com> wrote:
> I do believe KVGEN will meet your needs:
> https://drill.apache.org/docs/kvgen/
>
> On Mon, Sep 21, 2015 at 2:11 PM, John Omernik <jo...@omernik.com> wrote:
>
> > I have some poorly developed json where the developer used data for key
> > names
> >
> > {"created":"2015-12-01", "ZYS":"BLAH"}
> > {"created":"2015-12-01", "ZYX":"BLAH"}
> > {"created":"2015-12-01", "ABC":"BLAH"}
> > {"created":"2015-12-01", "ADS":"BLAH"}
> >
> > I'd like to somehow map the key name to a value and give it a generic
> name
> >
> > select `created`, somemagic() as value1 from table
> >
> > Not sure how this would work, or if it's possible, or how I'd even
> > reference that, but thought I would ask.
> >
> > John
> >
>
>
>
> --
> *Jim Scott*
> Director, Enterprise Strategy & Architecture
> +1 (347) 746-9281
> @kingmesal <https://twitter.com/kingmesal>
>
> <http://www.mapr.com/>
> [image: MapR Technologies] <http://www.mapr.com>
>
> Now Available - Free Hadoop On-Demand Training
> <
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> >
>
Re: JSON Using "keys" as values
Posted by Jim Scott <js...@maprtech.com>.
I do believe KVGEN will meet your needs:
https://drill.apache.org/docs/kvgen/
On Mon, Sep 21, 2015 at 2:11 PM, John Omernik <jo...@omernik.com> wrote:
> I have some poorly developed json where the developer used data for key
> names
>
> {"created":"2015-12-01", "ZYS":"BLAH"}
> {"created":"2015-12-01", "ZYX":"BLAH"}
> {"created":"2015-12-01", "ABC":"BLAH"}
> {"created":"2015-12-01", "ADS":"BLAH"}
>
> I'd like to somehow map the key name to a value and give it a generic name
>
> select `created`, somemagic() as value1 from table
>
> Not sure how this would work, or if it's possible, or how I'd even
> reference that, but thought I would ask.
>
> John
>
--
*Jim Scott*
Director, Enterprise Strategy & Architecture
+1 (347) 746-9281
@kingmesal <https://twitter.com/kingmesal>
<http://www.mapr.com/>
[image: MapR Technologies] <http://www.mapr.com>
Now Available - Free Hadoop On-Demand Training
<http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available>