You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Sean Curtis <se...@gmail.com> on 2010/12/15 05:23:01 UTC
hive newbie - importing data into hive
just wondering if i have a pipe delimited file, how i can just import this data into hive:
basically i am using the microlens database, which is pipe separated. for example:
user id | age | gender | occupation | zip code
translates to
123 | 24 | M | worker | 12345
i'd like to just import this straight into Hive. my initial thoughts:
1. use unix substitute command and change all "|" to "Ctrl-A".
2. import into mysql, then use sqoop
seems it should be easier than this. can someone help?
thanks for the help.
sean
Re: hive newbie - importing data into hive
Posted by Kirk True <ki...@mustardgrain.com>.
Hi Sean,
The delimiter used is specified in the create table statement, not the
LOAD DATA statement.
Kirk
On 12/14/2010 08:59 PM, Sean Curtis wrote:
> thanks Bryan
>
> should have been more specific. checked that guide and it didnt seem
> obvious with LOAD INFILE how to take a file that was already
> pipe-delimited and straight import that.
>
> is there a section of the doc i may have missed or some tip that can
> help there?
>
> sean
>
> On Dec 14, 2010, at 11:36 PM, Bryan Talbot wrote:
>
>> I'll help by recommending that you get started by looking at the
>> "Getting Started Guide".
>>
>> http://wiki.apache.org/hadoop/Hive/GettingStarted
>>
>>
>> -Bryan
>>
>>
>> On Tue, Dec 14, 2010 at 8:23 PM, Sean Curtis <sean.curtis@gmail.com
>> <ma...@gmail.com>> wrote:
>>
>> just wondering if i have a pipe delimited file, how i can just
>> import this data into hive:
>>
>> basically i am using the microlens database, which is pipe
>> separated. for example:
>>
>> user id | age | gender | occupation | zip code
>>
>> translates to
>>
>> 123 | 24 | M | worker | 12345
>>
>>
>> i'd like to just import this straight into Hive. my initial thoughts:
>> 1. use unix substitute command and change all "|" to "Ctrl-A".
>> 2. import into mysql, then use sqoop
>>
>> seems it should be easier than this. can someone help?
>>
>> thanks for the help.
>>
>> sean
>>
>>
>
Re: hive newbie - importing data into hive
Posted by Sonal Goyal <so...@gmail.com>.
Sean,
You can refer to
http://wiki.apache.org/hadoop/Hive/LanguageManual/DDL#Create.2BAC8-Drop_Table
You can define the ROW FORMAT DELIMITED as part of the table definition and
then load your data into the table.
Thanks and Regards,
Sonal
<https://github.com/sonalgoyal/hiho>Connect Hadoop with databases,
Salesforce, FTP servers and others <https://github.com/sonalgoyal/hiho>
Nube Technologies <http://www.nubetech.co>
<http://in.linkedin.com/in/sonalgoyal>
On Wed, Dec 15, 2010 at 10:29 AM, Sean Curtis <se...@gmail.com> wrote:
> thanks Bryan
>
> should have been more specific. checked that guide and it didnt seem
> obvious with LOAD INFILE how to take a file that was already pipe-delimited
> and straight import that.
>
> is there a section of the doc i may have missed or some tip that can help
> there?
>
> sean
>
> On Dec 14, 2010, at 11:36 PM, Bryan Talbot wrote:
>
> I'll help by recommending that you get started by looking at the "Getting
> Started Guide".
>
> http://wiki.apache.org/hadoop/Hive/GettingStarted
>
>
> -Bryan
>
>
> On Tue, Dec 14, 2010 at 8:23 PM, Sean Curtis <se...@gmail.com>wrote:
>
>> just wondering if i have a pipe delimited file, how i can just import this
>> data into hive:
>>
>> basically i am using the microlens database, which is pipe separated. for
>> example:
>>
>> user id | age | gender | occupation | zip code
>>
>> translates to
>>
>> 123 | 24 | M | worker | 12345
>>
>>
>> i'd like to just import this straight into Hive. my initial thoughts:
>> 1. use unix substitute command and change all "|" to "Ctrl-A".
>> 2. import into mysql, then use sqoop
>>
>> seems it should be easier than this. can someone help?
>>
>> thanks for the help.
>>
>> sean
>
>
>
>
Re: hive newbie - importing data into hive
Posted by Sean Curtis <se...@gmail.com>.
thanks Bryan
should have been more specific. checked that guide and it didnt seem obvious with LOAD INFILE how to take a file that was already pipe-delimited and straight import that.
is there a section of the doc i may have missed or some tip that can help there?
sean
On Dec 14, 2010, at 11:36 PM, Bryan Talbot wrote:
> I'll help by recommending that you get started by looking at the "Getting Started Guide".
>
> http://wiki.apache.org/hadoop/Hive/GettingStarted
>
>
> -Bryan
>
>
> On Tue, Dec 14, 2010 at 8:23 PM, Sean Curtis <se...@gmail.com> wrote:
> just wondering if i have a pipe delimited file, how i can just import this data into hive:
>
> basically i am using the microlens database, which is pipe separated. for example:
>
> user id | age | gender | occupation | zip code
>
> translates to
>
> 123 | 24 | M | worker | 12345
>
>
> i'd like to just import this straight into Hive. my initial thoughts:
> 1. use unix substitute command and change all "|" to "Ctrl-A".
> 2. import into mysql, then use sqoop
>
> seems it should be easier than this. can someone help?
>
> thanks for the help.
>
> sean
>
Re: hive newbie - importing data into hive
Posted by Bryan Talbot <bt...@aeriagames.com>.
I'll help by recommending that you get started by looking at the "Getting
Started Guide".
http://wiki.apache.org/hadoop/Hive/GettingStarted
-Bryan
On Tue, Dec 14, 2010 at 8:23 PM, Sean Curtis <se...@gmail.com> wrote:
> just wondering if i have a pipe delimited file, how i can just import this
> data into hive:
>
> basically i am using the microlens database, which is pipe separated. for
> example:
>
> user id | age | gender | occupation | zip code
>
> translates to
>
> 123 | 24 | M | worker | 12345
>
>
> i'd like to just import this straight into Hive. my initial thoughts:
> 1. use unix substitute command and change all "|" to "Ctrl-A".
> 2. import into mysql, then use sqoop
>
> seems it should be easier than this. can someone help?
>
> thanks for the help.
>
> sean