You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by ylyy-1985 <yl...@163.com> on 2012/04/13 15:24:20 UTC

to change hive delimiter->'||'

hi, all. I see something strange. the defaule delimiter is \001 and I want to change to '||'. So I just run my command :

CREATE TABLE table1(
channelid INT,serviceno STRING,regid STRING,pageurl STRING,urlext STRING,pagetitle STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '||' stored as textfile;

then I load the txt data(with one record="12||12003||2131||44123||22222||112") into the table, select * from the table, I see that the 6 fields are filled with

"12","","12003","","2131","" 

I think the empty value is "\t",which replace the "||", am I right? I think there is some other ways to replace the delimiter and make hive work fine(of cause we can overwrite the inputformat class).


2012-04-13



Best Regards
Andes

Email:ylyy-1985@163.com

Re: to change hive delimiter->'||'

Posted by Edward Capriolo <ed...@gmail.com>.
You can not have a '||' with the delimted serde. Just insert dummy
columns in your DML.

id - name
5 || bob

create table people ( id int, dummy1 string , name string)


On Fri, Apr 13, 2012 at 9:30 AM, Tucker, Matt <Ma...@disney.com> wrote:
> I’m not sure how multiple-character delimiters work with Hive, but you might
> try escaping each pipe (“\|\|”).  I had a similar issue with using pipes in
> str_to_map().
>
> Matt Tucker
>
> From: ylyy-1985 [mailto:ylyy-1985@163.com]
> Sent: Friday, April 13, 2012 9:24 AM
> To: hive_user
> Subject: to change hive delimiter->'||'
>
>
>
> hi, all. I see something strange. the defaule delimiter is \001 and I want
> to change to '||'. So I just run my command :
>
>
>
> CREATE TABLE table1(
> channelid INT,serviceno STRING,regid STRING,pageurl STRING,urlext
> STRING,pagetitle STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '||'
> stored as textfile;
>
>
>
> then I load the txt data(with one
> record="12||12003||2131||44123||22222||112") into the table, select * from
> the table, I see that the 6 fields are filled with
>
>
>
> "12","","12003","","2131",""
>
>
>
> I think the empty value is "\t",which replace the "||", am I right? I think
> there is some other ways to replace the delimiter and make hive work fine(of
> cause we can overwrite the inputformat class).
>
>
>
>
>
> 2012-04-13
>
> ________________________________
>
> Best Regards
>
> Andes
>
>
>
> Email:ylyy-1985@163.com

回复: RE: to change hive delimiter->'||'

Posted by ylyy-1985 <yl...@163.com>.
I had tried this, but not work. and the sys works with '|'. 

2012-04-13



Best Regards
Andes

Email:ylyy-1985@163.com



发件人：Tucker, Matt
发送时间：2012-04-13 21:32
主题：RE: to change hive delimiter->'||'
收件人："user@hive.apache.org"<us...@hive.apache.org>
抄送：

I’m not sure how multiple-character delimiters work with Hive, but you might try escaping each pipe (“\|\|”).  I had a similar issue with using pipes in str_to_map().
Matt Tucker
From: ylyy-1985 [mailto:ylyy-1985@163.com] 
Sent: Friday, April 13, 2012 9:24 AM
To: hive_user
Subject: to change hive delimiter->'||'

hi, all. I see something strange. the defaule delimiter is \001 and I want to change to '||'. So I just run my command :

CREATE TABLE table1(
channelid INT,serviceno STRING,regid STRING,pageurl STRING,urlext STRING,pagetitle STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '||' stored as textfile;

then I load the txt data(with one record="12||12003||2131||44123||22222||112") into the table, select * from the table, I see that the 6 fields are filled with

"12","","12003","","2131","" 

I think the empty value is "\t",which replace the "||", am I right? I think there is some other ways to replace the delimiter and make hive work fine(of cause we can overwrite the inputformat class).


2012-04-13



Best Regards
Andes

Email:ylyy-1985@163.com

RE: to change hive delimiter->'||'

Posted by "Tucker, Matt" <Ma...@disney.com>.
I’m not sure how multiple-character delimiters work with Hive, but you might try escaping each pipe (“\|\|”).  I had a similar issue with using pipes in str_to_map().
Matt Tucker
From: ylyy-1985 [mailto:ylyy-1985@163.com]
Sent: Friday, April 13, 2012 9:24 AM
To: hive_user
Subject: to change hive delimiter->'||'

hi, all. I see something strange. the defaule delimiter is \001 and I want to change to '||'. So I just run my command :

CREATE TABLE table1(
channelid INT,serviceno STRING,regid STRING,pageurl STRING,urlext STRING,pagetitle STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '||' stored as textfile;

then I load the txt data(with one record="12||12003||2131||44123||22222||112") into the table, select * from the table, I see that the 6 fields are filled with

"12","","12003","","2131",""

I think the empty value is "\t",which replace the "||", am I right? I think there is some other ways to replace the delimiter and make hive work fine(of cause we can overwrite the inputformat class).


2012-04-13
________________________________
Best Regards
Andes

Email:ylyy-1985@163.com<ma...@163.com>