You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by daidong <da...@gmail.com> on 2014/01/27 18:14:54 UTC
HBase MapReduce with setup function problem
Dear all,
I am writing a MapReduce application processing HBase table. In each map,
it needs to read data from another HBase table, so i use the 'setup'
function to initialize the HTable instance like this:
@Override
public void setup(Context context){
Configuration conf = HBaseConfiguration.create();
try {
centrals = new HTable(conf, "central".getBytes());
} catch (IOException e) {
}
return;
}
But, when i run this mapreduce application, it is always stay in 0%,0%. And
the map phase is always under initializing, does not progress. I have
googled it, but still do not have any ideas.
P.S. I use hadoop-1.1.2 and Hbase-0.96.
Thanks!
- Dong
Re: HBase MapReduce with setup function problem
Posted by Ted Yu <yu...@gmail.com>.
I agree that we should find the cause for why initialization got stuck.
I noticed empty catch block:
} catch (IOException e) {
}
Can you add some logging there to see what might have gone wrong ?
Thanks
On Mon, Jan 27, 2014 at 11:56 AM, daidong <da...@gmail.com> wrote:
> Dear Ted, Thanks very much for your reply!
>
> Yes. MultiTableInputFormat may work here, but i still want to know how to
> connect a hbase table inside MapReduce applications. Because i may need
> also write to tables inside map function.
>
> Do you know why previous mr application does not work? Because the wrong
> configuration instance? Any suggestion will be great for me! Thanks!
>
> - Dong
>
>
> 2014-01-27 Ted Yu <yu...@gmail.com>
>
> > Have you considered using MultiTableInputFormat ?
> >
> > Cheers
> >
> >
> > On Mon, Jan 27, 2014 at 9:14 AM, daidong <da...@gmail.com> wrote:
> >
> > > Dear all,
> > >
> > > I am writing a MapReduce application processing HBase table. In each
> > map,
> > > it needs to read data from another HBase table, so i use the 'setup'
> > > function to initialize the HTable instance like this:
> > >
> > > @Override
> > >
> > > public void setup(Context context){
> > >
> > > Configuration conf = HBaseConfiguration.create();
> > >
> > > try {
> > >
> > > centrals = new HTable(conf, "central".getBytes());
> > >
> > > } catch (IOException e) {
> > >
> > > }
> > >
> > > return;
> > >
> > > }
> > >
> > > But, when i run this mapreduce application, it is always stay in 0%,0%.
> > And
> > > the map phase is always under initializing, does not progress. I have
> > > googled it, but still do not have any ideas.
> > >
> > > P.S. I use hadoop-1.1.2 and Hbase-0.96.
> > >
> > > Thanks!
> > >
> > > - Dong
> > >
> >
>
Re: HBase MapReduce with setup function problem
Posted by daidong <da...@gmail.com>.
Dear Ted, Thanks very much for your reply!
Yes. MultiTableInputFormat may work here, but i still want to know how to
connect a hbase table inside MapReduce applications. Because i may need
also write to tables inside map function.
Do you know why previous mr application does not work? Because the wrong
configuration instance? Any suggestion will be great for me! Thanks!
- Dong
2014-01-27 Ted Yu <yu...@gmail.com>
> Have you considered using MultiTableInputFormat ?
>
> Cheers
>
>
> On Mon, Jan 27, 2014 at 9:14 AM, daidong <da...@gmail.com> wrote:
>
> > Dear all,
> >
> > I am writing a MapReduce application processing HBase table. In each
> map,
> > it needs to read data from another HBase table, so i use the 'setup'
> > function to initialize the HTable instance like this:
> >
> > @Override
> >
> > public void setup(Context context){
> >
> > Configuration conf = HBaseConfiguration.create();
> >
> > try {
> >
> > centrals = new HTable(conf, "central".getBytes());
> >
> > } catch (IOException e) {
> >
> > }
> >
> > return;
> >
> > }
> >
> > But, when i run this mapreduce application, it is always stay in 0%,0%.
> And
> > the map phase is always under initializing, does not progress. I have
> > googled it, but still do not have any ideas.
> >
> > P.S. I use hadoop-1.1.2 and Hbase-0.96.
> >
> > Thanks!
> >
> > - Dong
> >
>
Re: HBase MapReduce with setup function problem
Posted by Ted Yu <yu...@gmail.com>.
Have you considered using MultiTableInputFormat ?
Cheers
On Mon, Jan 27, 2014 at 9:14 AM, daidong <da...@gmail.com> wrote:
> Dear all,
>
> I am writing a MapReduce application processing HBase table. In each map,
> it needs to read data from another HBase table, so i use the 'setup'
> function to initialize the HTable instance like this:
>
> @Override
>
> public void setup(Context context){
>
> Configuration conf = HBaseConfiguration.create();
>
> try {
>
> centrals = new HTable(conf, "central".getBytes());
>
> } catch (IOException e) {
>
> }
>
> return;
>
> }
>
> But, when i run this mapreduce application, it is always stay in 0%,0%. And
> the map phase is always under initializing, does not progress. I have
> googled it, but still do not have any ideas.
>
> P.S. I use hadoop-1.1.2 and Hbase-0.96.
>
> Thanks!
>
> - Dong
>