You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by kanwal <ka...@gmail.com> on 2014/04/09 21:27:37 UTC
HBase Unable to find Region Server - No Exception being thrown
I'm currently running into an issue on my local setup where my application is
unable to connect to the hbase table but I'm successfully able to query the
table using hbase shell.
I'm using HTable client to make the connection and would expect to get an
error after certain retries when it's unable to establish connection.
However I'm seeing the code is continuously retrying and not logging any
error exception. I had to turn on debug to find the issue.
Is there a setting that we could use to throw an exception after certain
number of retries?
countersTable = new HTable(hbaseConfig, counters)
Using HBase Version - 0.94.15
14-04-09 12:11:36 DEBUG HConnectionManager$HConnectionImplementation:1083 -
locateRegionInMeta parentTable=.META., metaLocation=null, attempt=0 of 10
failed; retrying after sleep of 1000 because: Unable to find region for
counter,,99999999999999 after 10 tries.
14-04-09 12:11:37 DEBUG ZKUtil:1597 - hconnection-0x14547b13745000e
Retrieved 50 byte(s) of data from znode /hbase/root-region-server and set
watcher; mseakdang.corp.service-now.co...
14-04-09 12:11:37 DEBUG HConnectionManager$HConnectionImplementation:875 -
Looked up root region location,
connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@300b6421;
serverName=mseakdang.corp.service-now.com,60020,1397067169749
14-04-09 12:12:35 DEBUG HConnectionManager$HConnectionImplementation:1083 -
locateRegionInMeta parentTable=-ROOT-,
metaLocation={region=-ROOT-,,0.70236052,
hostname=mseakdang.corp.service-now.com, port=60020}, attempt=0 of 10
failed; retrying after sleep of 1002 because: unknown host:
mseakdang.corp.service-now.com
14-04-09 12:12:35 DEBUG ZKUtil:1597 - hconnection-0x14547b13745000e
Retrieved 50 byte(s) of data from znode /hbase/root-region-server and set
watcher; mseakdang.corp.service-now.co...
14-04-09 12:12:35 DEBUG HConnectionManager$HConnectionImplementation:875 -
Looked up root region location,
connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@300b6421;
serverName=mseakdang.corp.service-now.com,60020,1397067169749
14-04-09 12:12:36 DEBUG HConnectionManager$HConnectionImplementation:1083 -
locateRegionInMeta parentTable=.META., metaLocation=null, attempt=0 of 10
failed; retrying after sleep of 1008 because: Unable to find region for
counter,,99999999999999 after 10 tries.
14-04-09 12:12:37 DEBUG ZKUtil:1597 - hconnection-0x14547b13745000e
Retrieved 50 byte(s) of data from znode /hbase/root-region-server and set
watcher; mseakdang.corp.service-now.co...
14-04-09 12:12:37 DEBUG HConnectionManager$HConnectionImplementation:875 -
Looked up root region location,
connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@300b6421;
serverName=mseakdang.corp.service-now.com,60020,1397067169749
14-04-09 12:13:35 DEBUG HConnectionManager$HConnectionImplementation:1083 -
locateRegionInMeta parentTable=-ROOT-,
metaLocation={region=-ROOT-,,0.70236052,
hostname=mseakdang.corp.service-now.com, port=60020}, attempt=0 of 10
failed; retrying after sleep of 1001 because: unknown host:
mseakdang.corp.service-now.com
14-04-09 12:13:35 DEBUG ZKUtil:1597 - hconnection-0x14547b13745000e
Retrieved 50 byte(s) of data from znode /hbase/root-region-server and set
watcher; mseakdang.corp.service-now.co...
14-04-09 12:13:35 DEBUG HConnectionManager$HConnectionImplementation:875 -
Looked up root region location,
connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@300b6421;
serverName=mseakdang.corp.service-now.com,60020,1397067169749
14-04-09 12:13:36 DEBUG HConnectionManager$HConnectionImplementation:1083 -
locateRegionInMeta parentTable=.META., metaLocation=null, attempt=1 of 10
failed; retrying after sleep of 1000 because: Unable to find region for
counter,,99999999999999 after 10 tries.
--
View this message in context: http://apache-hbase.679495.n3.nabble.com/HBase-Unable-to-find-Region-Server-No-Exception-being-thrown-tp4058033.html
Sent from the HBase User mailing list archive at Nabble.com.
Re: HBase Unable to find Region Server - No Exception being thrown
Posted by Asaf Mesika <as...@gmail.com>.
There is a property you can tune to lower default num of retries from 10 to
any number like 2.
On Wednesday, April 9, 2014, kanwal <ka...@gmail.com> wrote:
> I'm currently running into an issue on my local setup where my application
> is
> unable to connect to the hbase table but I'm successfully able to query the
> table using hbase shell.
>
> I'm using HTable client to make the connection and would expect to get an
> error after certain retries when it's unable to establish connection.
> However I'm seeing the code is continuously retrying and not logging any
> error exception. I had to turn on debug to find the issue.
>
> Is there a setting that we could use to throw an exception after certain
> number of retries?
>
> countersTable = new HTable(hbaseConfig, counters)
>
> Using HBase Version - 0.94.15
>
> 14-04-09 12:11:36 DEBUG HConnectionManager$HConnectionImplementation:1083 -
> locateRegionInMeta parentTable=.META., metaLocation=null, attempt=0 of 10
> failed; retrying after sleep of 1000 because: Unable to find region for
> counter,,99999999999999 after 10 tries.
> 14-04-09 12:11:37 DEBUG ZKUtil:1597 - hconnection-0x14547b13745000e
> Retrieved 50 byte(s) of data from znode /hbase/root-region-server and set
> watcher; mseakdang.corp.service-now.co...
> 14-04-09 12:11:37 DEBUG HConnectionManager$HConnectionImplementation:875 -
> Looked up root region location,
>
> connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@300b6421
> ;
> serverName=mseakdang.corp.service-now.com,60020,1397067169749
> 14-04-09 12:12:35 DEBUG HConnectionManager$HConnectionImplementation:1083 -
> locateRegionInMeta parentTable=-ROOT-,
> metaLocation={region=-ROOT-,,0.70236052,
> hostname=mseakdang.corp.service-now.com, port=60020}, attempt=0 of 10
> failed; retrying after sleep of 1002 because: unknown host:
> mseakdang.corp.service-now.com
> 14-04-09 12:12:35 DEBUG ZKUtil:1597 - hconnection-0x14547b13745000e
> Retrieved 50 byte(s) of data from znode /hbase/root-region-server and set
> watcher; mseakdang.corp.service-now.co...
> 14-04-09 12:12:35 DEBUG HConnectionManager$HConnectionImplementation:875 -
> Looked up root region location,
>
> connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@300b6421
> ;
> serverName=mseakdang.corp.service-now.com,60020,1397067169749
> 14-04-09 12:12:36 DEBUG HConnectionManager$HConnectionImplementation:1083 -
> locateRegionInMeta parentTable=.META., metaLocation=null, attempt=0 of 10
> failed; retrying after sleep of 1008 because: Unable to find region for
> counter,,99999999999999 after 10 tries.
> 14-04-09 12:12:37 DEBUG ZKUtil:1597 - hconnection-0x14547b13745000e
> Retrieved 50 byte(s) of data from znode /hbase/root-region-server and set
> watcher; mseakdang.corp.service-now.co...
> 14-04-09 12:12:37 DEBUG HConnectionManager$HConnectionImplementation:875 -
> Looked up root region location,
>
> connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@300b6421
> ;
> serverName=mseakdang.corp.service-now.com,60020,1397067169749
> 14-04-09 12:13:35 DEBUG HConnectionManager$HConnectionImplementation:1083 -
> locateRegionInMeta parentTable=-ROOT-,
> metaLocation={region=-ROOT-,,0.70236052,
> hostname=mseakdang.corp.service-now.com, port=60020}, attempt=0 of 10
> failed; retrying after sleep of 1001 because: unknown host:
> mseakdang.corp.service-now.com
> 14-04-09 12:13:35 DEBUG ZKUtil:1597 - hconnection-0x14547b13745000e
> Retrieved 50 byte(s) of data from znode /hbase/root-region-server and set
> watcher; mseakdang.corp.service-now.co...
> 14-04-09 12:13:35 DEBUG HConnectionManager$HConnectionImplementation:875 -
> Looked up root region location,
>
> connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@300b6421
> ;
> serverName=mseakdang.corp.service-now.com,60020,1397067169749
> 14-04-09 12:13:36 DEBUG HConnectionManager$HConnectionImplementation:1083 -
> locateRegionInMeta parentTable=.META., metaLocation=null, attempt=1 of 10
> failed; retrying after sleep of 1000 because: Unable to find region for
> counter,,99999999999999 after 10 tries.
>
>
>
> --
> View this message in context:
> http://apache-hbase.679495.n3.nabble.com/HBase-Unable-to-find-Region-Server-No-Exception-being-thrown-tp4058033.html
> Sent from the HBase User mailing list archive at Nabble.com.
>
Re: HBase Unable to find Region Server - No Exception being thrown
Posted by kanwal <ka...@gmail.com>.
No. Couldn't find any error in the log.
On Wed, Apr 9, 2014 at 10:26 PM, Shengjun Xin [via Apache HBase] <
ml-node+s679495n4058051h23@n3.nabble.com> wrote:
> Does hbase-regionserver log have some error message?
>
>
> On Thu, Apr 10, 2014 at 3:27 AM, kanwal <[hidden email]<http://user/SendEmail.jtp?type=node&node=4058051&i=0>>
> wrote:
>
> > I'm currently running into an issue on my local setup where my
> application
> > is
> > unable to connect to the hbase table but I'm successfully able to query
> the
> > table using hbase shell.
> >
> > I'm using HTable client to make the connection and would expect to get
> an
> > error after certain retries when it's unable to establish connection.
> > However I'm seeing the code is continuously retrying and not logging any
> > error exception. I had to turn on debug to find the issue.
> >
> > Is there a setting that we could use to throw an exception after certain
> > number of retries?
> >
> > countersTable = new HTable(hbaseConfig, counters)
> >
> > Using HBase Version - 0.94.15
> >
> > 14-04-09 12:11:36 DEBUG
> HConnectionManager$HConnectionImplementation:1083 -
> > locateRegionInMeta parentTable=.META., metaLocation=null, attempt=0 of
> 10
> > failed; retrying after sleep of 1000 because: Unable to find region for
> > counter,,99999999999999 after 10 tries.
> > 14-04-09 12:11:37 DEBUG ZKUtil:1597 - hconnection-0x14547b13745000e
> > Retrieved 50 byte(s) of data from znode /hbase/root-region-server and
> set
> > watcher; mseakdang.corp.service-now.co...
> > 14-04-09 12:11:37 DEBUG HConnectionManager$HConnectionImplementation:875
> -
> > Looked up root region location,
> >
> >
> connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@300b6421
> > ;
> > serverName=mseakdang.corp.service-now.com,60020,1397067169749
> > 14-04-09 12:12:35 DEBUG
> HConnectionManager$HConnectionImplementation:1083 -
> > locateRegionInMeta parentTable=-ROOT-,
> > metaLocation={region=-ROOT-,,0.70236052,
> > hostname=mseakdang.corp.service-now.com, port=60020}, attempt=0 of 10
> > failed; retrying after sleep of 1002 because: unknown host:
> > mseakdang.corp.service-now.com
> > 14-04-09 12:12:35 DEBUG ZKUtil:1597 - hconnection-0x14547b13745000e
> > Retrieved 50 byte(s) of data from znode /hbase/root-region-server and
> set
> > watcher; mseakdang.corp.service-now.co...
> > 14-04-09 12:12:35 DEBUG HConnectionManager$HConnectionImplementation:875
> -
> > Looked up root region location,
> >
> >
> connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@300b6421
> > ;
> > serverName=mseakdang.corp.service-now.com,60020,1397067169749
> > 14-04-09 12:12:36 DEBUG
> HConnectionManager$HConnectionImplementation:1083 -
> > locateRegionInMeta parentTable=.META., metaLocation=null, attempt=0 of
> 10
> > failed; retrying after sleep of 1008 because: Unable to find region for
> > counter,,99999999999999 after 10 tries.
> > 14-04-09 12:12:37 DEBUG ZKUtil:1597 - hconnection-0x14547b13745000e
> > Retrieved 50 byte(s) of data from znode /hbase/root-region-server and
> set
> > watcher; mseakdang.corp.service-now.co...
> > 14-04-09 12:12:37 DEBUG HConnectionManager$HConnectionImplementation:875
> -
> > Looked up root region location,
> >
> >
> connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@300b6421
> > ;
> > serverName=mseakdang.corp.service-now.com,60020,1397067169749
> > 14-04-09 12:13:35 DEBUG
> HConnectionManager$HConnectionImplementation:1083 -
> > locateRegionInMeta parentTable=-ROOT-,
> > metaLocation={region=-ROOT-,,0.70236052,
> > hostname=mseakdang.corp.service-now.com, port=60020}, attempt=0 of 10
> > failed; retrying after sleep of 1001 because: unknown host:
> > mseakdang.corp.service-now.com
> > 14-04-09 12:13:35 DEBUG ZKUtil:1597 - hconnection-0x14547b13745000e
> > Retrieved 50 byte(s) of data from znode /hbase/root-region-server and
> set
> > watcher; mseakdang.corp.service-now.co...
> > 14-04-09 12:13:35 DEBUG HConnectionManager$HConnectionImplementation:875
> -
> > Looked up root region location,
> >
> >
> connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@300b6421
> > ;
> > serverName=mseakdang.corp.service-now.com,60020,1397067169749
> > 14-04-09 12:13:36 DEBUG
> HConnectionManager$HConnectionImplementation:1083 -
> > locateRegionInMeta parentTable=.META., metaLocation=null, attempt=1 of
> 10
> > failed; retrying after sleep of 1000 because: Unable to find region for
> > counter,,99999999999999 after 10 tries.
> >
> >
> >
> > --
> > View this message in context:
> >
> http://apache-hbase.679495.n3.nabble.com/HBase-Unable-to-find-Region-Server-No-Exception-being-thrown-tp4058033.html
> > Sent from the HBase User mailing list archive at Nabble.com.
> >
>
>
>
> --
> Regards
> Shengjun
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://apache-hbase.679495.n3.nabble.com/HBase-Unable-to-find-Region-Server-No-Exception-being-thrown-tp4058033p4058051.html
> To unsubscribe from HBase Unable to find Region Server - No Exception
> being thrown, click here<http://apache-hbase.679495.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4058033&code=a2Fud2FsMjM5QGdtYWlsLmNvbXw0MDU4MDMzfC0xMDIyNzMwOTE1>
> .
> NAML<http://apache-hbase.679495.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://apache-hbase.679495.n3.nabble.com/HBase-Unable-to-find-Region-Server-No-Exception-being-thrown-tp4058033p4058052.html
Sent from the HBase User mailing list archive at Nabble.com.
Re: HBase Unable to find Region Server - No Exception being thrown
Posted by Shengjun Xin <sx...@gopivotal.com>.
Does hbase-regionserver log have some error message?
On Thu, Apr 10, 2014 at 3:27 AM, kanwal <ka...@gmail.com> wrote:
> I'm currently running into an issue on my local setup where my application
> is
> unable to connect to the hbase table but I'm successfully able to query the
> table using hbase shell.
>
> I'm using HTable client to make the connection and would expect to get an
> error after certain retries when it's unable to establish connection.
> However I'm seeing the code is continuously retrying and not logging any
> error exception. I had to turn on debug to find the issue.
>
> Is there a setting that we could use to throw an exception after certain
> number of retries?
>
> countersTable = new HTable(hbaseConfig, counters)
>
> Using HBase Version - 0.94.15
>
> 14-04-09 12:11:36 DEBUG HConnectionManager$HConnectionImplementation:1083 -
> locateRegionInMeta parentTable=.META., metaLocation=null, attempt=0 of 10
> failed; retrying after sleep of 1000 because: Unable to find region for
> counter,,99999999999999 after 10 tries.
> 14-04-09 12:11:37 DEBUG ZKUtil:1597 - hconnection-0x14547b13745000e
> Retrieved 50 byte(s) of data from znode /hbase/root-region-server and set
> watcher; mseakdang.corp.service-now.co...
> 14-04-09 12:11:37 DEBUG HConnectionManager$HConnectionImplementation:875 -
> Looked up root region location,
>
> connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@300b6421
> ;
> serverName=mseakdang.corp.service-now.com,60020,1397067169749
> 14-04-09 12:12:35 DEBUG HConnectionManager$HConnectionImplementation:1083 -
> locateRegionInMeta parentTable=-ROOT-,
> metaLocation={region=-ROOT-,,0.70236052,
> hostname=mseakdang.corp.service-now.com, port=60020}, attempt=0 of 10
> failed; retrying after sleep of 1002 because: unknown host:
> mseakdang.corp.service-now.com
> 14-04-09 12:12:35 DEBUG ZKUtil:1597 - hconnection-0x14547b13745000e
> Retrieved 50 byte(s) of data from znode /hbase/root-region-server and set
> watcher; mseakdang.corp.service-now.co...
> 14-04-09 12:12:35 DEBUG HConnectionManager$HConnectionImplementation:875 -
> Looked up root region location,
>
> connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@300b6421
> ;
> serverName=mseakdang.corp.service-now.com,60020,1397067169749
> 14-04-09 12:12:36 DEBUG HConnectionManager$HConnectionImplementation:1083 -
> locateRegionInMeta parentTable=.META., metaLocation=null, attempt=0 of 10
> failed; retrying after sleep of 1008 because: Unable to find region for
> counter,,99999999999999 after 10 tries.
> 14-04-09 12:12:37 DEBUG ZKUtil:1597 - hconnection-0x14547b13745000e
> Retrieved 50 byte(s) of data from znode /hbase/root-region-server and set
> watcher; mseakdang.corp.service-now.co...
> 14-04-09 12:12:37 DEBUG HConnectionManager$HConnectionImplementation:875 -
> Looked up root region location,
>
> connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@300b6421
> ;
> serverName=mseakdang.corp.service-now.com,60020,1397067169749
> 14-04-09 12:13:35 DEBUG HConnectionManager$HConnectionImplementation:1083 -
> locateRegionInMeta parentTable=-ROOT-,
> metaLocation={region=-ROOT-,,0.70236052,
> hostname=mseakdang.corp.service-now.com, port=60020}, attempt=0 of 10
> failed; retrying after sleep of 1001 because: unknown host:
> mseakdang.corp.service-now.com
> 14-04-09 12:13:35 DEBUG ZKUtil:1597 - hconnection-0x14547b13745000e
> Retrieved 50 byte(s) of data from znode /hbase/root-region-server and set
> watcher; mseakdang.corp.service-now.co...
> 14-04-09 12:13:35 DEBUG HConnectionManager$HConnectionImplementation:875 -
> Looked up root region location,
>
> connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@300b6421
> ;
> serverName=mseakdang.corp.service-now.com,60020,1397067169749
> 14-04-09 12:13:36 DEBUG HConnectionManager$HConnectionImplementation:1083 -
> locateRegionInMeta parentTable=.META., metaLocation=null, attempt=1 of 10
> failed; retrying after sleep of 1000 because: Unable to find region for
> counter,,99999999999999 after 10 tries.
>
>
>
> --
> View this message in context:
> http://apache-hbase.679495.n3.nabble.com/HBase-Unable-to-find-Region-Server-No-Exception-being-thrown-tp4058033.html
> Sent from the HBase User mailing list archive at Nabble.com.
>
--
Regards
Shengjun