You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Chris Gong <fe...@163.com> on 2012/11/07 02:46:15 UTC
回复: hive integrate with hbase, map to existed hbase table report column family not exist
i got the reason, the column mapping section can't have any white space, including \r\n
Chris Gong
发件人： Chris Gong
发送时间： 2012-11-06 10:56
收件人： user-hive
主题： hive integrate with hbase, map to existed hbase table report column family not exist
hi all:
now, I'm map to an existed hbase table, i got the following infomation as:
FAILED: Error in metadata: MetaException(message:Column Family
data is not defined in hbase table df_money_files)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
my hive QL is
create external table hbase_money_files (rowkey string,
user_no string,
mon int,
mon_sn int,
group_no int,
sn int,
write_sect_no string,
write_sn int,
business_place_code string,
power_no int,
trans_group int,
price_code string,
ts_flag string,
elec_type_code string,
trade_type_code string,
ms_mode string,
user_ms_type string,
write_power double,
chg_power double,
add_power double,
kb_power double,
share_power double,
total_power double,
total_money double,
num_money double,
add_money1 double,
add_money2 double,
add_money3 double,
add_money4 double,
add_money5 double,
add_money6 double,
add_money7 double,
add_money8 double,
add_money9 double,
add_money10 double,
rp_power double,
rp_money double,
should_money double,
create_date string,
creator string,
warrant_no int,
line_code string,
trans_no string,
add_taxflag string,
write_date string,
compute_date string,
calculator_id string,
status string,
user_type1 string,
rela_user_no string,
part_sn int,
have_ext string,
id_fragment string,
check_date string,
check_man string,
start_date string)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,data:user_no,
data:mon,
data:mon_sn,
data:group_no,
data:sn,
data:write_sect_no,
data:write_sn,
data:business_place_code,
data:power_no,
data:trans_group,
data:price_code,
data:ts_flag,
data:elec_type_code,
data:trade_type_code,
data:ms_mode,
data:user_ms_type,
data:write_power,
data:chg_power,
data:add_power,
data:kb_power,
data:share_power,
data:total_power,
data:total_money,
data:num_money,
data:add_money1,
data:add_money2,
data:add_money3,
data:add_money4,
data:add_money5,
data:add_money6,
data:add_money7,
data:add_money8,
data:add_money9,
data:add_money10,
data:rp_power,
data:rp_money,
data:should_money,
data:create_date,
data:creator,
data:warrant_no,
data:line_code,
data:trans_no,
data:add_taxflag,
data:write_date,
data:compute_date,
data:calculator_id,
data:status,
data:user_type1,
data:rela_user_no,
data:part_sn,
data:have_ext,
data:id_fragment,
data:check_date,
data:check_man,
data:start_date")
TBLPROPERTIES("hbase.table.name" = "df_money_files");
however the data column does exist! when i describe the table in hbase shell, it reported:
hbase(main):001:0> describe 'df_money_files'
DESCRIPTION ENABLED
{NAME => 'df_money_files', FAMILIES => [{NAME => 'd true
ata', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '
0', VERSIONS => '3', COMPRESSION => 'NONE', MIN_VER
SIONS => '0', TTL => '2147483647', BLOCKSIZE => '65
536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}]}
1 row(s) in 0.8470 seconds
i am confused now, anyone can give some infomation?
Chris Gong
Re: hive integrate with hbase, map to existed hbase table report
column family not exist
Posted by Mark Grover <gr...@gmail.com>.
Indeed. https://issues.apache.org/jira/browse/HIVE-3243
Sorry you found out about it the hard way!
On Tue, Nov 6, 2012 at 5:46 PM, Chris Gong <fe...@163.com> wrote:
> **
> i got the reason, the column mapping section can't have any white space,
> including \r\n
>
> ------------------------------
> Chris Gong
>
> *发件人：* Chris Gong <fe...@163.com>
> *发送时间：* 2012-11-06 10:56
> *收件人：* user-hive <us...@hive.apache.org>
> *主题：* hive integrate with hbase, map to existed hbase table report column
> family not exist
> hi all:
> now, I'm map to an existed hbase table, i got the following infomation
> as:
>
>
> FAILED: Error in metadata: MetaException(message:Column Family
> data is not defined in hbase table df_money_files)
>
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
>
>
> my hive QL is
>
>
> create external table hbase_money_files (rowkey string,
> user_no string,
> mon int,
> mon_sn int,
> group_no int,
> sn int,
> write_sect_no string,
> write_sn int,
> business_place_code string,
> power_no int,
> trans_group int,
> price_code string,
> ts_flag string,
> elec_type_code string,
> trade_type_code string,
> ms_mode string,
> user_ms_type string,
> write_power double,
> chg_power double,
> add_power double,
> kb_power double,
> share_power double,
> total_power double,
> total_money double,
> num_money double,
> add_money1 double,
> add_money2 double,
> add_money3 double,
> add_money4 double,
> add_money5 double,
> add_money6 double,
> add_money7 double,
> add_money8 double,
> add_money9 double,
> add_money10 double,
> rp_power double,
> rp_money double,
> should_money double,
> create_date string,
> creator string,
> warrant_no int,
> line_code string,
> trans_no string,
> add_taxflag string,
> write_date string,
> compute_date string,
> calculator_id string,
> status string,
> user_type1 string,
> rela_user_no string,
> part_sn int,
> have_ext string,
> id_fragment string,
> check_date string,
> check_man string,
> start_date string)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,data:user_no,
> data:mon,
> data:mon_sn,
> data:group_no,
> data:sn,
> data:write_sect_no,
> data:write_sn,
> data:business_place_code,
> data:power_no,
> data:trans_group,
> data:price_code,
> data:ts_flag,
> data:elec_type_code,
> data:trade_type_code,
> data:ms_mode,
> data:user_ms_type,
> data:write_power,
> data:chg_power,
> data:add_power,
> data:kb_power,
> data:share_power,
> data:total_power,
> data:total_money,
> data:num_money,
> data:add_money1,
> data:add_money2,
> data:add_money3,
> data:add_money4,
> data:add_money5,
> data:add_money6,
> data:add_money7,
> data:add_money8,
> data:add_money9,
> data:add_money10,
> data:rp_power,
> data:rp_money,
> data:should_money,
> data:create_date,
> data:creator,
> data:warrant_no,
> data:line_code,
> data:trans_no,
> data:add_taxflag,
> data:write_date,
> data:compute_date,
> data:calculator_id,
> data:status,
> data:user_type1,
> data:rela_user_no,
> data:part_sn,
> data:have_ext,
> data:id_fragment,
> data:check_date,
> data:check_man,
> data:start_date")
> TBLPROPERTIES("hbase.table.name" = "df_money_files");
>
> however the data column does exist! when i describe the table in hbase
> shell, it reported:
>
>
> hbase(main):001:0> describe 'df_money_files'
> DESCRIPTION ENABLED
> {NAME => 'df_money_files', FAMILIES => [{NAME => 'd true
> ata', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '
> 0', VERSIONS => '3', COMPRESSION => 'NONE', MIN_VER
> SIONS => '0', TTL => '2147483647', BLOCKSIZE => '65
> 536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}]}
> 1 row(s) in 0.8470 seconds
>
> i am confused now, anyone can give some infomation?
> ------------------------------
> Chris Gong
>