You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Raj Hadoop <ha...@yahoo.com> on 2014/01/06 18:36:12 UTC
JSON data to HIVE table
Hi,
I am trying to load a data that is in JSON format to Hive table. Can any one suggest what is the method I need to follow?
Thanks,
Raj
Re: JSON data to HIVE table
Posted by Jay Vyas <ja...@gmail.com>.
One nice way to do this stuff is using a special SERDE, possible like the
JsonSerde:
A simpler scenario, where you have to load a multidelimiter CSV file, can
be addressed using the RegexSerde : which maps columns to each group
matching.
In your case, the JsonSerde could be used essentially to match your fields
and map them to columns.
For a simple example of how to use Regex Serdes to do the same for CSV
files: check out
https://github.com/jayunit100/bigpetstore/blob/master/src/main/java/org/bigtop/bigpetstore/etl/HiveETL.java
I *think* JsonSerde comes with HCatalog now, so if you dont have it on your
classpath, you can easily add that jar into hadoop/lib and then all your
mappers will be able to see your json data as tabular hive data.
Re: JSON data to HIVE table
Posted by Raj Hadoop <ha...@yahoo.com>.
All,
If I have to load JSON data to a Hive table (default record format while creating the table) - is that a requirement to convert each JSON record into one line.
How would I do this ?
Thanks,
Raj
________________________________
From: Rok Kralj <ro...@gmail.com>
To: user@hive.apache.org
Sent: Tuesday, January 7, 2014 3:54 AM
Subject: Re: JSON data to HIVE table
Also, if you have large or dynamic schemas which are a pain to write by hand, you can use this simple tool:
https://github.com/strelec/hive-serde-gen
2014/1/7 Roberto Congiu <ro...@openx.com>
Also https://github.com/rcongiu/Hive-JSON-Serde ;)
>
>
>
>On Mon, Jan 6, 2014 at 12:00 PM, Russell Jurney <ru...@gmail.com> wrote:
>
>Check these out:
>>
>>
>>http://hortonworks.com/blog/discovering-hive-schema-in-collections-of-json-documents/
>>
>>http://hortonworks.com/blog/howto-use-hive-to-sqlize-your-own-tweets-part-two-loading-hive-sql-queries/
>>
>>https://github.com/kevinweil/elephant-bird
>>
>>
>>
>>
>>On Mon, Jan 6, 2014 at 9:36 AM, Raj Hadoop <ha...@yahoo.com> wrote:
>>
>>Hi,
>>>
>>>I am trying to load a data that is in JSON format to Hive table. Can any one suggest what is the method I need to follow?
>>>
>>>Thanks,
>>>Raj
>>
>>
>>
>>--
>>Russell Jurney twitter.com/rjurney russell.jurney@gmail.com datasyndrome.com
>
>
>
>--
>----------------------------------------------------------
>Good judgement comes with experience.
>Experience comes with bad judgement.
>
>----------------------------------------------------------
>
>Roberto Congiu - Data Engineer - OpenX
>tel: +1 626 466 1141
--
eMail: rok.kralj@gmail.com
Re: JSON data to HIVE table
Posted by Rok Kralj <ro...@gmail.com>.
Also, if you have large or dynamic schemas which are a pain to write by
hand, you can use this simple tool:
https://github.com/strelec/hive-serde-gen
2014/1/7 Roberto Congiu <ro...@openx.com>
> Also https://github.com/rcongiu/Hive-JSON-Serde ;)
>
>
> On Mon, Jan 6, 2014 at 12:00 PM, Russell Jurney <ru...@gmail.com>wrote:
>
>> Check these out:
>>
>>
>> http://hortonworks.com/blog/discovering-hive-schema-in-collections-of-json-documents/
>>
>> http://hortonworks.com/blog/howto-use-hive-to-sqlize-your-own-tweets-part-two-loading-hive-sql-queries/
>> https://github.com/kevinweil/elephant-bird
>>
>>
>> On Mon, Jan 6, 2014 at 9:36 AM, Raj Hadoop <ha...@yahoo.com> wrote:
>>
>>> Hi,
>>>
>>> I am trying to load a data that is in JSON format to Hive table. Can any
>>> one suggest what is the method I need to follow?
>>>
>>> Thanks,
>>> Raj
>>>
>>
>>
>>
>> --
>> Russell Jurney twitter.com/rjurney russell.jurney@gmail.com datasyndrome.
>> com
>>
>
>
>
> --
> ----------------------------------------------------------
> Good judgement comes with experience.
> Experience comes with bad judgement.
> ----------------------------------------------------------
> Roberto Congiu - Data Engineer - OpenX
> tel: +1 626 466 1141
>
--
eMail: rok.kralj@gmail.com
Re: JSON data to HIVE table
Posted by Roberto Congiu <ro...@openx.com>.
Also https://github.com/rcongiu/Hive-JSON-Serde ;)
On Mon, Jan 6, 2014 at 12:00 PM, Russell Jurney <ru...@gmail.com>wrote:
> Check these out:
>
>
> http://hortonworks.com/blog/discovering-hive-schema-in-collections-of-json-documents/
>
> http://hortonworks.com/blog/howto-use-hive-to-sqlize-your-own-tweets-part-two-loading-hive-sql-queries/
> https://github.com/kevinweil/elephant-bird
>
>
> On Mon, Jan 6, 2014 at 9:36 AM, Raj Hadoop <ha...@yahoo.com> wrote:
>
>> Hi,
>>
>> I am trying to load a data that is in JSON format to Hive table. Can any
>> one suggest what is the method I need to follow?
>>
>> Thanks,
>> Raj
>>
>
>
>
> --
> Russell Jurney twitter.com/rjurney russell.jurney@gmail.com datasyndrome.
> com
>
--
----------------------------------------------------------
Good judgement comes with experience.
Experience comes with bad judgement.
----------------------------------------------------------
Roberto Congiu - Data Engineer - OpenX
tel: +1 626 466 1141
Re: JSON data to HIVE table
Posted by Russell Jurney <ru...@gmail.com>.
Check these out:
http://hortonworks.com/blog/discovering-hive-schema-in-collections-of-json-documents/
http://hortonworks.com/blog/howto-use-hive-to-sqlize-your-own-tweets-part-two-loading-hive-sql-queries/
https://github.com/kevinweil/elephant-bird
On Mon, Jan 6, 2014 at 9:36 AM, Raj Hadoop <ha...@yahoo.com> wrote:
> Hi,
>
> I am trying to load a data that is in JSON format to Hive table. Can any
> one suggest what is the method I need to follow?
>
> Thanks,
> Raj
>
--
Russell Jurney twitter.com/rjurney russell.jurney@gmail.com datasyndrome.com