You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Panayotis Antonopoulos <an...@hotmail.com> on 2011/05/01 03:30:46 UTC
RE: Help with NPE during bulk load (completebulkload)
You are right the -c flag doesn't work on CDH3U0 version.
-Dhbase.cluster.distributed=true solved the problem for the bulk upload.
However I am having the same NPE while trying to use the TableInputFormat.
Does anyone know what is going on?
Regards,
Panagiotis.
> From: andy.sautins@returnpath.net
> To: user@hbase.apache.org
> Date: Mon, 25 Apr 2011 09:50:17 -0600
> Subject: RE: Help with NPE during bulk load (completebulkload)
>
>
> Julio,
>
> We are running CDH3U0 ( hbase 0.90.1 ) so there may be some difference if you are running 0.90.2. Running your workaround on CDH3U0 I get the following:
>
> hadoop jar hbase-0.90.1-cdh3u0.jar completebulkload -c hbase-site.xml input table
> usage: completebulkload /path/to/hfileoutputformat-output tablename
>
> Note the above does not run and just prints out usage information.
>
> The following is a workaround that is currently working for us:
>
> hadoop jar hbase-0.90.1-cdh3u0.jar completebulkload -Dhbase.cluster.distributed=true input table
>
> Understood our workaround doesn't load the full hbase-site.xml but it seems to have been a sufficient workaround for our needs.
>
> So not sure if that's a difference between the 0.90.2 and CDH3U0, but thought it was worth mentioning.
>
> -----Original Message-----
> From: Julio Lopez [mailto:jclopez+hbase@andrew.cmu.edu]
> Sent: Monday, April 25, 2011 12:27 AM
> To: user@hbase.apache.org
> Subject: Re: Help with NPE during bulk load (completebulkload)
>
> Andy,
>
> What are the symptoms?
>
> You also need to include in your classpath the directory where the zookeeper config file (zoo.cfg) is located.
>
> Yes, HBASE-3714 addresses the issue discussed here. Although, it does not fully address the NPE in org.apache.hadoop.hbase.zookeeper.ZKConfig.parseZooCfg(ZKConfig.java:167) for the cases when either (a) no hbase-site.xml has been loaded, or (b) an incomplete configuration is provided, which does not specify a value for the hbase.cluster.distributed option.
>
> Cheers,
>
> -Julio
>
>
> Begin forwarded message:
>
> > Date: April 24, 2011 10:26:38 PM EDT
> > Subject: RE: Help with NPE during bulk load (completebulkload)
> > Source: user@hbase.apache.org Archives
> > Author: Andy Sautins
> >
> > Julio,
> >
> > I agree I would love to see your workaround documented. I quickly tried adding a '-c configfile'
> > flag to our completebulkload calls and wasn't able to get it to work.
> >
> > -----Original Message-----
> >
> > Sent: Sunday, April 24, 2011 5:39 PM
> > To: user@hbase.apache.org
> > Subject: Re: Help with NPE during bulk load (completebulkload)
> >
> > On Sun, Apr 24, 2011 at 4:29 PM, Andy Sautins <an...@returnpath.net> wrote:
> >> If I'm reading it right it looks like JIRA-3714 addresses this issue.
> >>
> > Yeah, it looks like https://issues.apache.org/jira/browse/HBASE-3714
> >
> >> Thanks for including your workaround ( adding -c flag ). That seems to be a better
> > workaround than the one we were using.
> >>
> >
> > We should add this to the documentation Julio? Mind providing example and where it should
> > go in the doc (I'll do the rest).
> >
> > Thanks,
> > St.Ack
> > Read more...
> >
>
Re: Help with NPE during bulk load (completebulkload)
Posted by Todd Lipcon <to...@cloudera.com>.
2011/4/30 Panayotis Antonopoulos <an...@hotmail.com>
>
> You are right the -c flag doesn't work on CDH3U0 version.
>
> -Dhbase.cluster.distributed=true solved the problem for the bulk upload.
>
> However I am having the same NPE while trying to use the TableInputFormat.
>
> Does anyone know what is going on?
>
Make sure that, when you pass a config into TableInputFormat, that you use
HBaseConfiguration.create() rather than new Configuration().
-Todd
>
> Regards,
> Panagiotis.
>
> > From: andy.sautins@returnpath.net
> > To: user@hbase.apache.org
> > Date: Mon, 25 Apr 2011 09:50:17 -0600
> > Subject: RE: Help with NPE during bulk load (completebulkload)
> >
> >
> > Julio,
> >
> > We are running CDH3U0 ( hbase 0.90.1 ) so there may be some difference
> if you are running 0.90.2. Running your workaround on CDH3U0 I get the
> following:
> >
> > hadoop jar hbase-0.90.1-cdh3u0.jar completebulkload -c hbase-site.xml
> input table
> > usage: completebulkload /path/to/hfileoutputformat-output tablename
> >
> > Note the above does not run and just prints out usage information.
> >
> > The following is a workaround that is currently working for us:
> >
> > hadoop jar hbase-0.90.1-cdh3u0.jar completebulkload
> -Dhbase.cluster.distributed=true input table
> >
> > Understood our workaround doesn't load the full hbase-site.xml but it
> seems to have been a sufficient workaround for our needs.
> >
> > So not sure if that's a difference between the 0.90.2 and CDH3U0, but
> thought it was worth mentioning.
> >
> > -----Original Message-----
> > From: Julio Lopez [mailto:jclopez+hbase@andrew.cmu.edu]
> > Sent: Monday, April 25, 2011 12:27 AM
> > To: user@hbase.apache.org
> > Subject: Re: Help with NPE during bulk load (completebulkload)
> >
> > Andy,
> >
> > What are the symptoms?
> >
> > You also need to include in your classpath the directory where the
> zookeeper config file (zoo.cfg) is located.
> >
> > Yes, HBASE-3714 addresses the issue discussed here. Although, it does
> not fully address the NPE in
> org.apache.hadoop.hbase.zookeeper.ZKConfig.parseZooCfg(ZKConfig.java:167)
> for the cases when either (a) no hbase-site.xml has been loaded, or (b) an
> incomplete configuration is provided, which does not specify a value for the
> hbase.cluster.distributed option.
> >
> > Cheers,
> >
> > -Julio
> >
> >
> > Begin forwarded message:
> >
> > > Date: April 24, 2011 10:26:38 PM EDT
> > > Subject: RE: Help with NPE during bulk load (completebulkload)
> > > Source: user@hbase.apache.org Archives
> > > Author: Andy Sautins
> > >
> > > Julio,
> > >
> > > I agree I would love to see your workaround documented. I quickly
> tried adding a '-c configfile'
> > > flag to our completebulkload calls and wasn't able to get it to work.
> > >
> > > -----Original Message-----
> > >
> > > Sent: Sunday, April 24, 2011 5:39 PM
> > > To: user@hbase.apache.org
> > > Subject: Re: Help with NPE during bulk load (completebulkload)
> > >
> > > On Sun, Apr 24, 2011 at 4:29 PM, Andy Sautins <
> andy.sautins@returnpath.net> wrote:
> > >> If I'm reading it right it looks like JIRA-3714 addresses this issue.
> > >>
> > > Yeah, it looks like https://issues.apache.org/jira/browse/HBASE-3714
> > >
> > >> Thanks for including your workaround ( adding -c flag ). That seems
> to be a better
> > > workaround than the one we were using.
> > >>
> > >
> > > We should add this to the documentation Julio? Mind providing example
> and where it should
> > > go in the doc (I'll do the rest).
> > >
> > > Thanks,
> > > St.Ack
> > > Read more...
> > >
> >
>
>
--
Todd Lipcon
Software Engineer, Cloudera