You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Suraj Varma <sv...@gmail.com> on 2010/03/13 23:55:17 UTC
Re: HBase remote client configuration experience
Hi J-D:
I wrote a patch for this (also attached) and wanted to get your thoughts
before submitting it.
Please see attached.
Thanks,
--Suraj
On Tue, Mar 9, 2010 at 10:27 AM, Jean-Daniel Cryans <jd...@apache.org>wrote:
> Totally agree, I created https://issues.apache.org/jira/browse/HBASE-2305
>
> J-D
>
> On Tue, Mar 9, 2010 at 8:13 AM, Doug Meil <do...@explorysmedical.com>
> wrote:
> > Hi Jeff-
> >
> > Thanks for responding. I agree that "hard-coding" is certainly not a
> good idea, but the intent of the post was to illustrate *which* minimal
> parameters were getting used to establish a remote connection (i.e., not
> necessarily *where*).
> >
> > One of the other points of this observation was a request for the
> defaulting of a zookeeper client port value which appears to be required
> either supplied via XML or in code. Personally, I'd like less wherever is
> reasonable.
> >
> >> FYI... If the last parameter isn't set you'll get the following
> error....
> >> could this be defaulted to a reasonable value?
> >
> >
> >
> >
> > -----Original Message-----
> > From: Jeff Zhang [mailto:zjffdu@gmail.com]
> > Sent: Tuesday, March 09, 2010 10:17 AM
> > To: hbase-user@hadoop.apache.org
> > Subject: Re: HBase remote client configuration experience
> >
> > Hi Doug,
> >
> > In my opinion, when using HBase Java API, you should put hbase-site.xml
> in
> > classpath. Set the parameter in hase-site.xml. Hard coding the parameters
> is
> > not a good idea.
> >
> >
> >
> > On Tue, Mar 9, 2010 at 7:09 AM, Doug Meil <doug.meil@explorysmedical.com
> >wrote:
> >
> >> Greetings-
> >>
> >> The question of HBase client remote configuration (i.e., the client is
> not
> >> running on the same machine as the HBase master) comes up periodically
> on
> >> these dist-lists and a common theme is a lack of documentation on
> exactly
> >> what it takes to configure a remote client connection.
> >>
> >> After some trial and error, this seems to be the absolute minimum number
> of
> >> parameters required for connecting to a remote HBase instance (tested
> with
> >> HBase 0.20.3).
> >>
> >> HBaseConfiguration config = new HBaseConfiguration();
> >> config.clear();
> >> config.set("hbase.zookeeper.quorum", "server");
> >> config.set("hbase.zookeeper.property.clientPort","2181");
> >>
> >> This is consistent with other references to the importance of
> >> "hbase.zookeeper.quorum" except that we did this testing with the intent
> of
> >> connecting using only parameters set in code to eliminate any classpath
> >> issues with XML files.
> >>
> >> FYI... If the last parameter isn't set you'll get the following
> error....
> >> could this be defaulted to a reasonable value?
> >>
> >>
> >> 10/03/09 10:04:55 ERROR zookeeper.ZooKeeperWrapper: no clientPort found
> in
> >> zoo.cfg
> >> Exception in thread "main" java.io.IOException: Could not read quorum
> >> servers from zoo.cfg
> >> at
> >>
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.<init>(ZooKeeperWrapper.java:85)
> >> at
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$ClientZKWatcher.getZooKeeperWrapper(HConnectionManager.java:223)
> >> at
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getZooKeeperWrapper(HConnectionManager.java:932)
> >> at
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRootRegion(HConnectionManager.java:948)
> >> at
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:625)
> >>
> >>
> >> Doug Meil
> >> Director of Engineering
> >> doug.meil@explorys.net
> >>
> >>
> >
> >
> > --
> > Best Regards
> >
> > Jeff Zhang
> >
>
Re: HBase remote client configuration experience
Posted by Suraj Varma <sv...@gmail.com>.
Oye - ok ... will do.
--Suraj
On Sat, Mar 13, 2010 at 2:59 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
> Suraj,
>
> The mailing list doesn't accept attachments 99% of the time, might as
> well post it in the jira ;)
>
> J-D
>
> On Sat, Mar 13, 2010 at 2:55 PM, Suraj Varma <sv...@gmail.com> wrote:
> > Hi J-D:
> > I wrote a patch for this (also attached) and wanted to get your thoughts
> > before submitting it.
> > Please see attached.
> > Thanks,
> > --Suraj
> > On Tue, Mar 9, 2010 at 10:27 AM, Jean-Daniel Cryans <jdcryans@apache.org
> >
> > wrote:
> >>
> >> Totally agree, I created
> https://issues.apache.org/jira/browse/HBASE-2305
> >>
> >> J-D
> >>
> >> On Tue, Mar 9, 2010 at 8:13 AM, Doug Meil <
> doug.meil@explorysmedical.com>
> >> wrote:
> >> > Hi Jeff-
> >> >
> >> > Thanks for responding. I agree that "hard-coding" is certainly not a
> >> > good idea, but the intent of the post was to illustrate *which*
> minimal
> >> > parameters were getting used to establish a remote connection (i.e.,
> not
> >> > necessarily *where*).
> >> >
> >> > One of the other points of this observation was a request for the
> >> > defaulting of a zookeeper client port value which appears to be
> required
> >> > either supplied via XML or in code. Personally, I'd like less
> wherever is
> >> > reasonable.
> >> >
> >> >> FYI... If the last parameter isn't set you'll get the following
> >> >> error....
> >> >> could this be defaulted to a reasonable value?
> >> >
> >> >
> >> >
> >> >
> >> > -----Original Message-----
> >> > From: Jeff Zhang [mailto:zjffdu@gmail.com]
> >> > Sent: Tuesday, March 09, 2010 10:17 AM
> >> > To: hbase-user@hadoop.apache.org
> >> > Subject: Re: HBase remote client configuration experience
> >> >
> >> > Hi Doug,
> >> >
> >> > In my opinion, when using HBase Java API, you should put
> hbase-site.xml
> >> > in
> >> > classpath. Set the parameter in hase-site.xml. Hard coding the
> >> > parameters is
> >> > not a good idea.
> >> >
> >> >
> >> >
> >> > On Tue, Mar 9, 2010 at 7:09 AM, Doug Meil
> >> > <do...@explorysmedical.com>wrote:
> >> >
> >> >> Greetings-
> >> >>
> >> >> The question of HBase client remote configuration (i.e., the client
> is
> >> >> not
> >> >> running on the same machine as the HBase master) comes up
> periodically
> >> >> on
> >> >> these dist-lists and a common theme is a lack of documentation on
> >> >> exactly
> >> >> what it takes to configure a remote client connection.
> >> >>
> >> >> After some trial and error, this seems to be the absolute minimum
> >> >> number of
> >> >> parameters required for connecting to a remote HBase instance (tested
> >> >> with
> >> >> HBase 0.20.3).
> >> >>
> >> >> HBaseConfiguration config = new HBaseConfiguration();
> >> >> config.clear();
> >> >> config.set("hbase.zookeeper.quorum", "server");
> >> >> config.set("hbase.zookeeper.property.clientPort","2181");
> >> >>
> >> >> This is consistent with other references to the importance of
> >> >> "hbase.zookeeper.quorum" except that we did this testing with the
> >> >> intent of
> >> >> connecting using only parameters set in code to eliminate any
> classpath
> >> >> issues with XML files.
> >> >>
> >> >> FYI... If the last parameter isn't set you'll get the following
> >> >> error....
> >> >> could this be defaulted to a reasonable value?
> >> >>
> >> >>
> >> >> 10/03/09 10:04:55 ERROR zookeeper.ZooKeeperWrapper: no clientPort
> found
> >> >> in
> >> >> zoo.cfg
> >> >> Exception in thread "main" java.io.IOException: Could not read quorum
> >> >> servers from zoo.cfg
> >> >> at
> >> >>
> >> >>
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.<init>(ZooKeeperWrapper.java:85)
> >> >> at
> >> >>
> >> >>
> org.apache.hadoop.hbase.client.HConnectionManager$ClientZKWatcher.getZooKeeperWrapper(HConnectionManager.java:223)
> >> >> at
> >> >>
> >> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getZooKeeperWrapper(HConnectionManager.java:932)
> >> >> at
> >> >>
> >> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRootRegion(HConnectionManager.java:948)
> >> >> at
> >> >>
> >> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:625)
> >> >>
> >> >>
> >> >> Doug Meil
> >> >> Director of Engineering
> >> >> doug.meil@explorys.net
> >> >>
> >> >>
> >> >
> >> >
> >> > --
> >> > Best Regards
> >> >
> >> > Jeff Zhang
> >> >
> >
> >
>
Re: HBase remote client configuration experience
Posted by Jean-Daniel Cryans <jd...@apache.org>.
Suraj,
The mailing list doesn't accept attachments 99% of the time, might as
well post it in the jira ;)
J-D
On Sat, Mar 13, 2010 at 2:55 PM, Suraj Varma <sv...@gmail.com> wrote:
> Hi J-D:
> I wrote a patch for this (also attached) and wanted to get your thoughts
> before submitting it.
> Please see attached.
> Thanks,
> --Suraj
> On Tue, Mar 9, 2010 at 10:27 AM, Jean-Daniel Cryans <jd...@apache.org>
> wrote:
>>
>> Totally agree, I created https://issues.apache.org/jira/browse/HBASE-2305
>>
>> J-D
>>
>> On Tue, Mar 9, 2010 at 8:13 AM, Doug Meil <do...@explorysmedical.com>
>> wrote:
>> > Hi Jeff-
>> >
>> > Thanks for responding. I agree that "hard-coding" is certainly not a
>> > good idea, but the intent of the post was to illustrate *which* minimal
>> > parameters were getting used to establish a remote connection (i.e., not
>> > necessarily *where*).
>> >
>> > One of the other points of this observation was a request for the
>> > defaulting of a zookeeper client port value which appears to be required
>> > either supplied via XML or in code. Personally, I'd like less wherever is
>> > reasonable.
>> >
>> >> FYI... If the last parameter isn't set you'll get the following
>> >> error....
>> >> could this be defaulted to a reasonable value?
>> >
>> >
>> >
>> >
>> > -----Original Message-----
>> > From: Jeff Zhang [mailto:zjffdu@gmail.com]
>> > Sent: Tuesday, March 09, 2010 10:17 AM
>> > To: hbase-user@hadoop.apache.org
>> > Subject: Re: HBase remote client configuration experience
>> >
>> > Hi Doug,
>> >
>> > In my opinion, when using HBase Java API, you should put hbase-site.xml
>> > in
>> > classpath. Set the parameter in hase-site.xml. Hard coding the
>> > parameters is
>> > not a good idea.
>> >
>> >
>> >
>> > On Tue, Mar 9, 2010 at 7:09 AM, Doug Meil
>> > <do...@explorysmedical.com>wrote:
>> >
>> >> Greetings-
>> >>
>> >> The question of HBase client remote configuration (i.e., the client is
>> >> not
>> >> running on the same machine as the HBase master) comes up periodically
>> >> on
>> >> these dist-lists and a common theme is a lack of documentation on
>> >> exactly
>> >> what it takes to configure a remote client connection.
>> >>
>> >> After some trial and error, this seems to be the absolute minimum
>> >> number of
>> >> parameters required for connecting to a remote HBase instance (tested
>> >> with
>> >> HBase 0.20.3).
>> >>
>> >> HBaseConfiguration config = new HBaseConfiguration();
>> >> config.clear();
>> >> config.set("hbase.zookeeper.quorum", "server");
>> >> config.set("hbase.zookeeper.property.clientPort","2181");
>> >>
>> >> This is consistent with other references to the importance of
>> >> "hbase.zookeeper.quorum" except that we did this testing with the
>> >> intent of
>> >> connecting using only parameters set in code to eliminate any classpath
>> >> issues with XML files.
>> >>
>> >> FYI... If the last parameter isn't set you'll get the following
>> >> error....
>> >> could this be defaulted to a reasonable value?
>> >>
>> >>
>> >> 10/03/09 10:04:55 ERROR zookeeper.ZooKeeperWrapper: no clientPort found
>> >> in
>> >> zoo.cfg
>> >> Exception in thread "main" java.io.IOException: Could not read quorum
>> >> servers from zoo.cfg
>> >> at
>> >>
>> >> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.<init>(ZooKeeperWrapper.java:85)
>> >> at
>> >>
>> >> org.apache.hadoop.hbase.client.HConnectionManager$ClientZKWatcher.getZooKeeperWrapper(HConnectionManager.java:223)
>> >> at
>> >>
>> >> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getZooKeeperWrapper(HConnectionManager.java:932)
>> >> at
>> >>
>> >> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRootRegion(HConnectionManager.java:948)
>> >> at
>> >>
>> >> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:625)
>> >>
>> >>
>> >> Doug Meil
>> >> Director of Engineering
>> >> doug.meil@explorys.net
>> >>
>> >>
>> >
>> >
>> > --
>> > Best Regards
>> >
>> > Jeff Zhang
>> >
>
>