You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Dino Kečo <di...@gmail.com> on 2011/12/19 16:20:20 UTC
TableInputFormat and TableOutputFormat are not using hbase resource
from cloud configuration
Hi HBase users,
I have been using HBase CDH3u0(0.90.1) for data storage in my project, and
I have been using Hadoop jobs to process data in HBase table. Strange thing
that I have noticed is that hbase-site.xml and hbase-default.xml where not
read from Hadoop class path. I downloaded HBase and Hadoop source code and
I noticed that HBaseConfiguration.addHbaseResources(getConf()); is never
called by TableInputFormat or TableOutputFormat and because of that
hbase-site.xml resource wasn't loaded into Configuration.
In my job I always use *org.apache.hadoop.conf.Configuration *class. I know
I can manually call HBaseConfiguration.addHbaseResources(getConf()); method
but why is this needed. I think this code should be part of TableInput and
TableOutput formats ?
Regards,
Dino K
Re: TableInputFormat and TableOutputFormat are not using hbase resource
from cloud configuration
Posted by Jan Lukavský <ja...@firma.seznam.cz>.
Hi Dino,
the configuraion is loaded by call to
TableMapReduceUtil.initTable{Mapper,Reducer}Job(), which you should
probably use to setup your jobs.
Regards,
Jan
On 20.12.2011 08:56, Dino Kečo wrote:
> Hi Stack,
>
> I am using ToolRunner for running my jobs, so I have implemented Tool
> interface and extend Configured class and those classes create
> Configuration for me automatically.
>
> Because of that I need to add one line of code (HBaseConfiguration.
> addHbaseResources(getConf());) for loading HBase resources which is not
> that straight forward.
>
> It is not problem for me to add that but it is not that user friendly and
> users expect that those resources are loaded but they are not. So my
> question is should this be changed on HBase side ?
>
> Regards,
> Dino
Re: TableInputFormat and TableOutputFormat are not using hbase
resource from cloud configuration
Posted by Dino Kečo <di...@gmail.com>.
Hi Stack,
I am using ToolRunner for running my jobs, so I have implemented Tool
interface and extend Configured class and those classes create
Configuration for me automatically.
Because of that I need to add one line of code (HBaseConfiguration.
addHbaseResources(getConf());) for loading HBase resources which is not
that straight forward.
It is not problem for me to add that but it is not that user friendly and
users expect that those resources are loaded but they are not. So my
question is should this be changed on HBase side ?
Regards,
Dino
Re: TableInputFormat and TableOutputFormat are not using hbase
resource from cloud configuration
Posted by Stack <st...@duboce.net>.
On Mon, Dec 19, 2011 at 7:20 AM, Dino Kečo <di...@gmail.com> wrote:
> Hi HBase users,
>
> I have been using HBase CDH3u0(0.90.1) for data storage in my project, and
> I have been using Hadoop jobs to process data in HBase table. Strange thing
> that I have noticed is that hbase-site.xml and hbase-default.xml where not
> read from Hadoop class path. I downloaded HBase and Hadoop source code and
> I noticed that HBaseConfiguration.addHbaseResources(getConf()); is never
> called by TableInputFormat or TableOutputFormat and because of that
> hbase-site.xml resource wasn't loaded into Configuration.
>
>
> In my job I always use *org.apache.hadoop.conf.Configuration *class. I know
> I can manually call HBaseConfiguration.addHbaseResources(getConf()); method
> but why is this needed. I think this code should be part of TableInput and
> TableOutput formats ?
>
Setting up your job, try making a Configuration by doing
HBaseConfiguration.create()?
St.Ack