You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by al...@aim.com on 2012/10/04 20:30:00 UTC
no data in external table
Hello,
I use hive-0.9.0 with hadoop-0.20.2 and hbase -0.92.1. I have created external table, mapping it to an existing table in hbase. When I do "select * from myextrenaltable" it returns no results, although scan in hbase shows data, and I do not see any errors in jobtracker log.
Any ideas how to debug this issue.
Thanks.
Alex.
Re: no data in external table
Posted by al...@aim.com.
Hi,
I upgraded to hadoop-1.0.3, but still have the same results.
I did as you suggested. When I do "select * from hbase_table_2" it does not output any job information. When I do "select value from hbase_table_2" it outputs job information. In both cases there is no data returned.
Thanks.
Alex.
-----Original Message-----
From: kulkarni.swarnim <ku...@gmail.com>
To: user <us...@hive.apache.org>
Sent: Thu, Oct 4, 2012 3:16 pm
Subject: Re: no data in external table
Can you try creating a table like this:
CREATE EXTERNAL TABLE hbase_table_2(key int, value string)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
TBLPROPERTIES ("hbase.table.name" = "xyz");
Now do a select * from hbase_table_2;
Do you see any data now?
On Thu, Oct 4, 2012 at 5:10 PM, <al...@aim.com> wrote:
Hi,
In the hbase table I do not see column qualifier, only family.
For testing connection to hbase I also created a table using
CREATE TABLE hbase_table_1(key int, value string)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
TBLPROPERTIES ("hbase.table.name" = "xyz");
I see xyz table in hbase. then I added a row in hbase using put 'xyz', 'row1', 'cf1', 'abc'
Then in hive I did: select * from hbase_table_1;
No results are returned, but scan xys in hbase returns 1 row.
Thanks.
Alex.
-----Original Message-----
From: kulkarni.swarnim <ku...@gmail.com>
To: user <us...@hive.apache.org>
Sent: Thu, Oct 4, 2012 3:00 pm
Subject: Re: no data in external table
> "hbase.columns.mapping" = ":key,mtdt:string,il:string,ol:string"
This doesn't look right. The mapping should be of form COLUMN_FAMILY:COLUMN_QUALIFIER. In this case it seems to be COLUMN_FAMILY:TYPE which is not right.
On Thu, Oct 4, 2012 at 3:25 PM, <al...@aim.com> wrote:
Hi,
In hive shell I did
create external table myextrenaltable (key string, metadata string, inlinks string, outlinks string) stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
with serdeproperties ("hbase.columns.mapping" = ":key,mtdt:string,il:string,ol:string")
tblproperties ("hbase.table.name" = "myextrenaltable");
In tasktracker log I do not see anything relevant to hbase. In jobdetails page I see a few successful jobs. in hive shell I see
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201210031146_0016, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201210031146_0016
Kill Command = /home/dev/hadoop-0.20.2/bin/../bin/hadoop job -Dmapred.job.tracker=localhost:9001 -kill job_201210031146_0016
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
2012-10-04 13:19:06,581 Stage-1 map = 0%, reduce = 0%
2012-10-04 13:19:12,629 Stage-1 map = 100%, reduce = 0%
2012-10-04 13:19:15,657 Stage-1 map = 100%, reduce = 100%
Ended Job = job_201210031146_0016
MapReduce Jobs Launched:
Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 SUCCESS
Total MapReduce CPU Time Spent: 0 msec
OK
Time taken: 17.47 seconds
Thanks in advance.
Alex.
-----Original Message-----
From: Ted Yu <yu...@gmail.com>
To: user <us...@hive.apache.org>
Sent: Thu, Oct 4, 2012 11:33 am
Subject: Re: no data in external table
Can you tell us how you created mapping for the existing table ?
In task log, do you see any connection attempt to HBase ?
Cheers
On Thu, Oct 4, 2012 at 11:30 AM, <al...@aim.com> wrote:
Hello,
I use hive-0.9.0 with hadoop-0.20.2 and hbase -0.92.1. I have created external table, mapping it to an existing table in hbase. When I do "select * from myextrenaltable" it returns no results, although scan in hbase shows data, and I do not see any errors in jobtracker log.
Any ideas how to debug this issue.
Thanks.
Alex.
--
Swarnim
--
Swarnim
Re: no data in external table
Posted by "kulkarni.swarnim@gmail.com" <ku...@gmail.com>.
Can you try creating a table like this:
CREATE EXTERNAL TABLE hbase_table_2(key int, value string)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
TBLPROPERTIES ("hbase.table.name" = "xyz");
Now do a select * from hbase_table_2;
Do you see any data now?
On Thu, Oct 4, 2012 at 5:10 PM, <al...@aim.com> wrote:
> Hi,
>
> In the hbase table I do not see column qualifier, only family.
> For testing connection to hbase I also created a table using
>
> CREATE TABLE hbase_table_1(key int, value string)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
> TBLPROPERTIES ("hbase.table.name" = "xyz");
>
> I see xyz table in hbase. then I added a row in hbase using put 'xyz', 'row1', 'cf1', 'abc'
>
>
> Then in hive I did: select * from hbase_table_1;
> No results are returned, but scan xys in hbase returns 1 row.
>
> Thanks.
>
> Alex.
>
> -----Original Message-----
> From: kulkarni.swarnim <ku...@gmail.com>
> To: user <us...@hive.apache.org>
> Sent: Thu, Oct 4, 2012 3:00 pm
> Subject: Re: no data in external table
> > "hbase.columns.mapping" = ":key,mtdt:string,il:string,ol:string"
>
> This doesn't look right. The mapping should be of form
> COLUMN_FAMILY:COLUMN_QUALIFIER. In this case it seems to be
> COLUMN_FAMILY:TYPE which is not right.
>
> On Thu, Oct 4, 2012 at 3:25 PM, <al...@aim.com> wrote:
>
>> Hi,
>>
>> In hive shell I did
>>
>> create external table myextrenaltable (key string, metadata string,
>> inlinks string, outlinks string) stored by
>> 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>> with serdeproperties ("hbase.columns.mapping" =
>> ":key,mtdt:string,il:string,ol:string")
>> tblproperties ("hbase.table.name" = "myextrenaltable");
>>
>> In tasktracker log I do not see anything relevant to hbase. In jobdetails
>> page I see a few successful jobs. in hive shell I see
>>
>> Total MapReduce jobs = 1
>> Launching Job 1 out of 1
>> Number of reduce tasks is set to 0 since there's no reduce operator
>> Starting Job = job_201210031146_0016, Tracking URL =
>> http://localhost:50030/jobdetails.jsp?jobid=job_201210031146_0016
>> Kill Command = /home/dev/hadoop-0.20.2/bin/../bin/hadoop job
>> -Dmapred.job.tracker=localhost:9001 -kill job_201210031146_0016
>> Hadoop job information for Stage-1: number of mappers: 1; number of
>> reducers: 0
>> 2012-10-04 13:19:06,581 Stage-1 map = 0%, reduce = 0%
>> 2012-10-04 13:19:12,629 Stage-1 map = 100%, reduce = 0%
>> 2012-10-04 13:19:15,657 Stage-1 map = 100%, reduce = 100%
>> Ended Job = job_201210031146_0016
>> MapReduce Jobs Launched:
>> Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 SUCCESS
>> Total MapReduce CPU Time Spent: 0 msec
>> OK
>> Time taken: 17.47 seconds
>>
>>
>>
>> Thanks in advance.
>> Alex.
>>
>> -----Original Message-----
>> From: Ted Yu <yu...@gmail.com>
>> To: user <us...@hive.apache.org>
>> Sent: Thu, Oct 4, 2012 11:33 am
>> Subject: Re: no data in external table
>>
>> Can you tell us how you created mapping for the existing table ?
>>
>> In task log, do you see any connection attempt to HBase ?
>>
>> Cheers
>>
>> On Thu, Oct 4, 2012 at 11:30 AM, <al...@aim.com> wrote:
>>
>>> Hello,
>>>
>>> I use hive-0.9.0 with hadoop-0.20.2 and hbase -0.92.1. I have created
>>> external table, mapping it to an existing table in hbase. When I do "select
>>> * from myextrenaltable" it returns no results, although scan in hbase shows
>>> data, and I do not see any errors in jobtracker log.
>>>
>>> Any ideas how to debug this issue.
>>>
>>> Thanks.
>>> Alex.
>>>
>>
>>
>
>
> --
> Swarnim
>
--
Swarnim
Re: no data in external table
Posted by al...@aim.com.
Hi,
In the hbase table I do not see column qualifier, only family.
For testing connection to hbase I also created a table using
CREATE TABLE hbase_table_1(key int, value string)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
TBLPROPERTIES ("hbase.table.name" = "xyz");
I see xyz table in hbase. then I added a row in hbase using put 'xyz', 'row1', 'cf1', 'abc'
Then in hive I did: select * from hbase_table_1;
No results are returned, but scan xys in hbase returns 1 row.
Thanks.
Alex.
-----Original Message-----
From: kulkarni.swarnim <ku...@gmail.com>
To: user <us...@hive.apache.org>
Sent: Thu, Oct 4, 2012 3:00 pm
Subject: Re: no data in external table
> "hbase.columns.mapping" = ":key,mtdt:string,il:string,ol:string"
This doesn't look right. The mapping should be of form COLUMN_FAMILY:COLUMN_QUALIFIER. In this case it seems to be COLUMN_FAMILY:TYPE which is not right.
On Thu, Oct 4, 2012 at 3:25 PM, <al...@aim.com> wrote:
Hi,
In hive shell I did
create external table myextrenaltable (key string, metadata string, inlinks string, outlinks string) stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
with serdeproperties ("hbase.columns.mapping" = ":key,mtdt:string,il:string,ol:string")
tblproperties ("hbase.table.name" = "myextrenaltable");
In tasktracker log I do not see anything relevant to hbase. In jobdetails page I see a few successful jobs. in hive shell I see
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201210031146_0016, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201210031146_0016
Kill Command = /home/dev/hadoop-0.20.2/bin/../bin/hadoop job -Dmapred.job.tracker=localhost:9001 -kill job_201210031146_0016
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
2012-10-04 13:19:06,581 Stage-1 map = 0%, reduce = 0%
2012-10-04 13:19:12,629 Stage-1 map = 100%, reduce = 0%
2012-10-04 13:19:15,657 Stage-1 map = 100%, reduce = 100%
Ended Job = job_201210031146_0016
MapReduce Jobs Launched:
Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 SUCCESS
Total MapReduce CPU Time Spent: 0 msec
OK
Time taken: 17.47 seconds
Thanks in advance.
Alex.
-----Original Message-----
From: Ted Yu <yu...@gmail.com>
To: user <us...@hive.apache.org>
Sent: Thu, Oct 4, 2012 11:33 am
Subject: Re: no data in external table
Can you tell us how you created mapping for the existing table ?
In task log, do you see any connection attempt to HBase ?
Cheers
On Thu, Oct 4, 2012 at 11:30 AM, <al...@aim.com> wrote:
Hello,
I use hive-0.9.0 with hadoop-0.20.2 and hbase -0.92.1. I have created external table, mapping it to an existing table in hbase. When I do "select * from myextrenaltable" it returns no results, although scan in hbase shows data, and I do not see any errors in jobtracker log.
Any ideas how to debug this issue.
Thanks.
Alex.
--
Swarnim
Re: no data in external table
Posted by "kulkarni.swarnim@gmail.com" <ku...@gmail.com>.
> "hbase.columns.mapping" = ":key,mtdt:string,il:string,ol:string"
This doesn't look right. The mapping should be of form
COLUMN_FAMILY:COLUMN_QUALIFIER. In this case it seems to be
COLUMN_FAMILY:TYPE which is not right.
On Thu, Oct 4, 2012 at 3:25 PM, <al...@aim.com> wrote:
> Hi,
>
> In hive shell I did
>
> create external table myextrenaltable (key string, metadata string,
> inlinks string, outlinks string) stored by
> 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> with serdeproperties ("hbase.columns.mapping" =
> ":key,mtdt:string,il:string,ol:string")
> tblproperties ("hbase.table.name" = "myextrenaltable");
>
> In tasktracker log I do not see anything relevant to hbase. In jobdetails
> page I see a few successful jobs. in hive shell I see
>
> Total MapReduce jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks is set to 0 since there's no reduce operator
> Starting Job = job_201210031146_0016, Tracking URL =
> http://localhost:50030/jobdetails.jsp?jobid=job_201210031146_0016
> Kill Command = /home/dev/hadoop-0.20.2/bin/../bin/hadoop job
> -Dmapred.job.tracker=localhost:9001 -kill job_201210031146_0016
> Hadoop job information for Stage-1: number of mappers: 1; number of
> reducers: 0
> 2012-10-04 13:19:06,581 Stage-1 map = 0%, reduce = 0%
> 2012-10-04 13:19:12,629 Stage-1 map = 100%, reduce = 0%
> 2012-10-04 13:19:15,657 Stage-1 map = 100%, reduce = 100%
> Ended Job = job_201210031146_0016
> MapReduce Jobs Launched:
> Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 SUCCESS
> Total MapReduce CPU Time Spent: 0 msec
> OK
> Time taken: 17.47 seconds
>
>
>
> Thanks in advance.
> Alex.
>
> -----Original Message-----
> From: Ted Yu <yu...@gmail.com>
> To: user <us...@hive.apache.org>
> Sent: Thu, Oct 4, 2012 11:33 am
> Subject: Re: no data in external table
>
> Can you tell us how you created mapping for the existing table ?
>
> In task log, do you see any connection attempt to HBase ?
>
> Cheers
>
> On Thu, Oct 4, 2012 at 11:30 AM, <al...@aim.com> wrote:
>
>> Hello,
>>
>> I use hive-0.9.0 with hadoop-0.20.2 and hbase -0.92.1. I have created
>> external table, mapping it to an existing table in hbase. When I do "select
>> * from myextrenaltable" it returns no results, although scan in hbase shows
>> data, and I do not see any errors in jobtracker log.
>>
>> Any ideas how to debug this issue.
>>
>> Thanks.
>> Alex.
>>
>
>
--
Swarnim
Re: no data in external table
Posted by al...@aim.com.
Hi,
In hive shell I did
create external table myextrenaltable (key string, metadata string, inlinks string, outlinks string) stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
with serdeproperties ("hbase.columns.mapping" = ":key,mtdt:string,il:string,ol:string")
tblproperties ("hbase.table.name" = "myextrenaltable");
In tasktracker log I do not see anything relevant to hbase. In jobdetails page I see a few successful jobs. in hive shell I see
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201210031146_0016, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201210031146_0016
Kill Command = /home/dev/hadoop-0.20.2/bin/../bin/hadoop job -Dmapred.job.tracker=localhost:9001 -kill job_201210031146_0016
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
2012-10-04 13:19:06,581 Stage-1 map = 0%, reduce = 0%
2012-10-04 13:19:12,629 Stage-1 map = 100%, reduce = 0%
2012-10-04 13:19:15,657 Stage-1 map = 100%, reduce = 100%
Ended Job = job_201210031146_0016
MapReduce Jobs Launched:
Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 SUCCESS
Total MapReduce CPU Time Spent: 0 msec
OK
Time taken: 17.47 seconds
Thanks in advance.
Alex.
-----Original Message-----
From: Ted Yu <yu...@gmail.com>
To: user <us...@hive.apache.org>
Sent: Thu, Oct 4, 2012 11:33 am
Subject: Re: no data in external table
Can you tell us how you created mapping for the existing table ?
In task log, do you see any connection attempt to HBase ?
Cheers
On Thu, Oct 4, 2012 at 11:30 AM, <al...@aim.com> wrote:
Hello,
I use hive-0.9.0 with hadoop-0.20.2 and hbase -0.92.1. I have created external table, mapping it to an existing table in hbase. When I do "select * from myextrenaltable" it returns no results, although scan in hbase shows data, and I do not see any errors in jobtracker log.
Any ideas how to debug this issue.
Thanks.
Alex.
Re: no data in external table
Posted by Ted Yu <yu...@gmail.com>.
Can you tell us how you created mapping for the existing table ?
In task log, do you see any connection attempt to HBase ?
Cheers
On Thu, Oct 4, 2012 at 11:30 AM, <al...@aim.com> wrote:
> Hello,
>
> I use hive-0.9.0 with hadoop-0.20.2 and hbase -0.92.1. I have created
> external table, mapping it to an existing table in hbase. When I do "select
> * from myextrenaltable" it returns no results, although scan in hbase shows
> data, and I do not see any errors in jobtracker log.
>
> Any ideas how to debug this issue.
>
> Thanks.
> Alex.
>