You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by "Sunderlin, Mark" <ma...@teamaol.com> on 2011/03/02 15:27:58 UTC
Associative Arrays in Hive?
Let us say my log data that I want to place a log file into hive. And the log file itself looks something like this:
Event_time, event_type, event_data_blob
And the blob data looks like
"Key1=value1;key2=value2;key3=value3 ... keyn=valuen"
This looks like maybe I start like this:
Create table my_log(
Event_time STRING,
Event_type INT
Event_blob MAP<string,string>
How am I doing so far?
Then, I know I need to have some reformatting some on the blob to get it into the 'right' format, as I doubt the equals sign '=' works as a key to key value map in Hive, and I suspect the semi-colon ';' isn't what hive/hadoop wants for a key, value pair separator either.
What I don't know is what format to place the blob data into so I can then load it into a hive table.
Anyone got some pointers?
---
Mark E. Sunderlin
Solutions Architect |AOL Data Warehouse
P: 703-256-6935 | C: 540-327-6222
AIM: MESunderlin
22000 AOL Way | Dulles, VA | 20166
[aol-comp-logo]
Re: Associative Arrays in Hive?
Posted by 김영우 <wa...@gmail.com>.
Refer to this http://dev.bizo.com/2011/02/columns-in-hive.html
<http://dev.bizo.com/2011/02/columns-in-hive.html>HTH
- Youngwoo
2011/3/2 Sunderlin, Mark <ma...@teamaol.com>
> Let us say my log data that I want to place a log file into hive. And
> the log file itself looks something like this:
>
>
>
> Event_time, event_type, event_data_blob
>
>
>
> And the blob data looks like
>
> “Key1=value1;key2=value2;key3=value3 … keyn=valuen”
>
>
>
> This looks like maybe I start like this:
>
>
>
> Create table my_log(
>
> Event_time STRING,
>
> Event_type INT
>
> Event_blob MAP<string,string>
>
>
>
> How am I doing so far?
>
>
>
> Then, I know I need to have some reformatting some on the blob to get it
> into the ‘right’ format, as I doubt the equals sign ‘=’ works as a key to
> key value map in Hive, and I suspect the semi-colon ‘;’ isn’t what
> hive/hadoop wants for a key, value pair separator either.
>
>
>
> What I don’t know is what format to place the blob data into so I can then
> load it into a hive table.
>
>
>
> Anyone got some pointers?
>
>
>
> ---
>
> *Mark E. Sunderlin*
>
> *Solutions Architect **|AOL **Data Warehouse*
>
> P: 703-256-6935 | C: 540-327-6222
>
> AIM: MESunderlin
>
> 22000 AOL Way | Dulles, VA | 20166
>
> [image: aol-comp-logo]
>
>
>
Re: Associative Arrays in Hive?
Posted by Edward Capriolo <ed...@gmail.com>.
On Wed, Mar 2, 2011 at 9:27 AM, Sunderlin, Mark
<ma...@teamaol.com>wrote:
> Let us say my log data that I want to place a log file into hive. And
> the log file itself looks something like this:
>
>
>
> Event_time, event_type, event_data_blob
>
>
>
> And the blob data looks like
>
> “Key1=value1;key2=value2;key3=value3 … keyn=valuen”
>
>
>
> This looks like maybe I start like this:
>
>
>
> Create table my_log(
>
> Event_time STRING,
>
> Event_type INT
>
> Event_blob MAP<string,string>
>
>
>
> How am I doing so far?
>
>
>
> Then, I know I need to have some reformatting some on the blob to get it
> into the ‘right’ format, as I doubt the equals sign ‘=’ works as a key to
> key value map in Hive, and I suspect the semi-colon ‘;’ isn’t what
> hive/hadoop wants for a key, value pair separator either.
>
>
>
> What I don’t know is what format to place the blob data into so I can then
> load it into a hive table.
>
>
>
> Anyone got some pointers?
>
>
>
> ---
>
> *Mark E. Sunderlin*
>
> *Solutions Architect **|AOL **Data Warehouse*
>
> P: 703-256-6935 | C: 540-327-6222
>
> AIM: MESunderlin
>
> 22000 AOL Way | Dulles, VA | 20166
>
> [image: aol-comp-logo]
>
>
>
You can use = or any ascii character with a byte value less then 128.
The syntax when you create the table is:
row_format
: DELIMITED [FIELDS TERMINATED BY char] [COLLECTION ITEMS TERMINATED BY char]
[MAP KEYS TERMINATED BY char] [LINES TERMINATED BY char]
All hail the hive language manual.
http://wiki.apache.org/hadoop/Hive/LanguageManual/DDL#Create.2BAC8-Drop_Table