You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Akmal Abbasov <ak...@icloud.com> on 2015/09/03 15:24:33 UTC

HBase master WebUI is not responding

Hi,
I’m having problems with accessing HBase master webui. when I try to access it, it returns redirect page /master-status, and then it is freezes.
I’ve tried to wait for a long time, but still it didn’t work.
I am using hbase hbase-0.98.7-hadoop2, hadoop hadoop-2.5.1
Any suggestions?
Thanks.

Re: HBase master WebUI is not responding

Posted by Ted Yu <yu...@gmail.com>.
Just realized this discussion should be done on user@hbase mailing list.

Akmal:
If you still have questions, please start a thread there.

Thanks

On Fri, Sep 4, 2015 at 8:41 AM, Ted Yu <yu...@gmail.com> wrote:

> Can you do the following ?
>
> run hbck command to see if any inconsistency is reported
> check region server log on test-rs3 to see if hbase:meta is successfully
> opened
>
> Cheers
>
> On Fri, Sep 4, 2015 at 8:20 AM, Akmal Abbasov <ak...@icloud.com>
> wrote:
>
>> Hi Ted,
>>
>> No, I’ve restarted master node and now there is no exceptions, but still
>> webui is not responding
>> 2015-09-04 14:54:57,926 DEBUG [FifoRpcScheduler.handler1-thread-26]
>> master.AssignmentManager: based on AM, current
>> region=hbase:meta,,1.1588230740 is on server=test-rs3,60020,1441378286999
>> server being checked: test-rs5,60020,1441261824016
>> What else could be wrong?
>>
>> Thanks.
>>
>>
>>

Re: HBase master WebUI is not responding

Posted by Ted Yu <yu...@gmail.com>.
Just realized this discussion should be done on user@hbase mailing list.

Akmal:
If you still have questions, please start a thread there.

Thanks

On Fri, Sep 4, 2015 at 8:41 AM, Ted Yu <yu...@gmail.com> wrote:

> Can you do the following ?
>
> run hbck command to see if any inconsistency is reported
> check region server log on test-rs3 to see if hbase:meta is successfully
> opened
>
> Cheers
>
> On Fri, Sep 4, 2015 at 8:20 AM, Akmal Abbasov <ak...@icloud.com>
> wrote:
>
>> Hi Ted,
>>
>> No, I’ve restarted master node and now there is no exceptions, but still
>> webui is not responding
>> 2015-09-04 14:54:57,926 DEBUG [FifoRpcScheduler.handler1-thread-26]
>> master.AssignmentManager: based on AM, current
>> region=hbase:meta,,1.1588230740 is on server=test-rs3,60020,1441378286999
>> server being checked: test-rs5,60020,1441261824016
>> What else could be wrong?
>>
>> Thanks.
>>
>>
>>

Re: HBase master WebUI is not responding

Posted by Ted Yu <yu...@gmail.com>.
Just realized this discussion should be done on user@hbase mailing list.

Akmal:
If you still have questions, please start a thread there.

Thanks

On Fri, Sep 4, 2015 at 8:41 AM, Ted Yu <yu...@gmail.com> wrote:

> Can you do the following ?
>
> run hbck command to see if any inconsistency is reported
> check region server log on test-rs3 to see if hbase:meta is successfully
> opened
>
> Cheers
>
> On Fri, Sep 4, 2015 at 8:20 AM, Akmal Abbasov <ak...@icloud.com>
> wrote:
>
>> Hi Ted,
>>
>> No, I’ve restarted master node and now there is no exceptions, but still
>> webui is not responding
>> 2015-09-04 14:54:57,926 DEBUG [FifoRpcScheduler.handler1-thread-26]
>> master.AssignmentManager: based on AM, current
>> region=hbase:meta,,1.1588230740 is on server=test-rs3,60020,1441378286999
>> server being checked: test-rs5,60020,1441261824016
>> What else could be wrong?
>>
>> Thanks.
>>
>>
>>

Re: HBase master WebUI is not responding

Posted by Ted Yu <yu...@gmail.com>.
Just realized this discussion should be done on user@hbase mailing list.

Akmal:
If you still have questions, please start a thread there.

Thanks

On Fri, Sep 4, 2015 at 8:41 AM, Ted Yu <yu...@gmail.com> wrote:

> Can you do the following ?
>
> run hbck command to see if any inconsistency is reported
> check region server log on test-rs3 to see if hbase:meta is successfully
> opened
>
> Cheers
>
> On Fri, Sep 4, 2015 at 8:20 AM, Akmal Abbasov <ak...@icloud.com>
> wrote:
>
>> Hi Ted,
>>
>> No, I’ve restarted master node and now there is no exceptions, but still
>> webui is not responding
>> 2015-09-04 14:54:57,926 DEBUG [FifoRpcScheduler.handler1-thread-26]
>> master.AssignmentManager: based on AM, current
>> region=hbase:meta,,1.1588230740 is on server=test-rs3,60020,1441378286999
>> server being checked: test-rs5,60020,1441261824016
>> What else could be wrong?
>>
>> Thanks.
>>
>>
>>

Re: HBase master WebUI is not responding

Posted by Ted Yu <yu...@gmail.com>.
Can you do the following ?

run hbck command to see if any inconsistency is reported
check region server log on test-rs3 to see if hbase:meta is successfully
opened

Cheers

On Fri, Sep 4, 2015 at 8:20 AM, Akmal Abbasov <ak...@icloud.com>
wrote:

> Hi Ted,
>
> No, I’ve restarted master node and now there is no exceptions, but still
> webui is not responding
> 2015-09-04 14:54:57,926 DEBUG [FifoRpcScheduler.handler1-thread-26]
> master.AssignmentManager: based on AM, current
> region=hbase:meta,,1.1588230740 is on server=test-rs3,60020,1441378286999
> server being checked: test-rs5,60020,1441261824016
> What else could be wrong?
>
> Thanks.
>
>
> On 03 Sep 2015, at 16:20, Ted Yu <yu...@gmail.com> wrote:
>
> Did the exception lead to hbase:meta assignment failure ?
>
> Please check region server log on 10.10.8.53 <http://10.10.8.53:60000/>
>
> On Thu, Sep 3, 2015 at 7:15 AM, Akmal Abbasov <ak...@icloud.com>
> wrote:
>
>> Hi Ted,
>> I’m getting a lot of
>> 2015-09-03 14:08:20,651 WARN  [1865504699@qtp-838397579-13]
>> client.HConnectionManager$HConnectionImplementation: Checking master
>> connection
>> com.google.protobuf.ServiceException: java.net.SocketTimeoutException:
>> Call to test-m/10.10.8.53:60000 failed because
>> java.net.SocketTimeoutException: 60000 millis timeout while waiting for
>> channel to be ready for read. ch :
>> java.nio.channels.SocketChannel[connected local=/10.10.8.53:41243
>> remote=test-m/10.10.8.53:60000]
>>         at
>> org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1678)
>>         at
>> org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1719)
>>         at
>> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561)
>>         at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceState.isMasterRunning(HConnectionManager.java:1508)
>>         at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isKeepAliveMasterConnectedAndRunning(HConnectionManager.java:2146)
>>         at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1859)
>>         at
>> org.apache.hadoop.hbase.client.HBaseAdmin$MasterCallable.prepare(HBaseAdmin.java:3320)
>>         at
>> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:113)
>>         at
>> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90)
>>         at
>> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3347)
>>         at
>> org.apache.hadoop.hbase.client.HBaseAdmin.listTableDescriptorsByNamespace(HBaseAdmin.java:2281)
>>         at
>> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.__jamon_innerUnit__catalogTables(MasterStatusTmplImpl.java:465)
>>         at
>> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.renderNoFlush(MasterStatusTmplImpl.java:274)
>>         at
>> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.renderNoFlush(MasterStatusTmpl.java:386)
>>         at
>> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.render(MasterStatusTmpl.java:376)
>>         at
>> org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:95)
>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>>         at
>> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>>         at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
>>         at
>> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
>>         at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>         at
>> org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1122)
>>         at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>         at
>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>>         at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>         at java.io.FilterInputStream.read(FilterInputStream.java:133)
>>         at
>> org.apache.hadoop.hbase.ipc.RpcClient$Connection$PingInputStream.read(RpcClient.java:558)
>>         at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
>>         at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
>>         at java.io.DataInputStream.readInt(DataInputStream.java:387)
>>         at
>> org.apache.hadoop.hbase.ipc.RpcClient$Connection.readResponse(RpcClient.java:1076)
>>         at
>> org.apache.hadoop.hbase.ipc.RpcClient$Connection.run(RpcClient.java:727)
>>
>> Thanks
>>
>> > On 03 Sep 2015, at 15:50, Ted Yu <yu...@gmail.com> wrote:
>> >
>> > Can you log into the master node and check master log ?
>> >
>> > Pastebin snippet of master log if you need more help.
>> >
>> > Cheers
>> >
>> >
>> >> On Sep 3, 2015, at 6:24 AM, Akmal Abbasov <ak...@icloud.com>
>> wrote:
>> >>
>> >> Hi,
>> >> I’m having problems with accessing HBase master webui. when I try to
>> access it, it returns redirect page /master-status, and then it is freezes.
>> >> I’ve tried to wait for a long time, but still it didn’t work.
>> >> I am using hbase hbase-0.98.7-hadoop2, hadoop hadoop-2.5.1
>> >> Any suggestions?
>> >> Thanks.
>>
>>
>
>

Re: HBase master WebUI is not responding

Posted by Ted Yu <yu...@gmail.com>.
Can you do the following ?

run hbck command to see if any inconsistency is reported
check region server log on test-rs3 to see if hbase:meta is successfully
opened

Cheers

On Fri, Sep 4, 2015 at 8:20 AM, Akmal Abbasov <ak...@icloud.com>
wrote:

> Hi Ted,
>
> No, I’ve restarted master node and now there is no exceptions, but still
> webui is not responding
> 2015-09-04 14:54:57,926 DEBUG [FifoRpcScheduler.handler1-thread-26]
> master.AssignmentManager: based on AM, current
> region=hbase:meta,,1.1588230740 is on server=test-rs3,60020,1441378286999
> server being checked: test-rs5,60020,1441261824016
> What else could be wrong?
>
> Thanks.
>
>
> On 03 Sep 2015, at 16:20, Ted Yu <yu...@gmail.com> wrote:
>
> Did the exception lead to hbase:meta assignment failure ?
>
> Please check region server log on 10.10.8.53 <http://10.10.8.53:60000/>
>
> On Thu, Sep 3, 2015 at 7:15 AM, Akmal Abbasov <ak...@icloud.com>
> wrote:
>
>> Hi Ted,
>> I’m getting a lot of
>> 2015-09-03 14:08:20,651 WARN  [1865504699@qtp-838397579-13]
>> client.HConnectionManager$HConnectionImplementation: Checking master
>> connection
>> com.google.protobuf.ServiceException: java.net.SocketTimeoutException:
>> Call to test-m/10.10.8.53:60000 failed because
>> java.net.SocketTimeoutException: 60000 millis timeout while waiting for
>> channel to be ready for read. ch :
>> java.nio.channels.SocketChannel[connected local=/10.10.8.53:41243
>> remote=test-m/10.10.8.53:60000]
>>         at
>> org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1678)
>>         at
>> org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1719)
>>         at
>> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561)
>>         at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceState.isMasterRunning(HConnectionManager.java:1508)
>>         at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isKeepAliveMasterConnectedAndRunning(HConnectionManager.java:2146)
>>         at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1859)
>>         at
>> org.apache.hadoop.hbase.client.HBaseAdmin$MasterCallable.prepare(HBaseAdmin.java:3320)
>>         at
>> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:113)
>>         at
>> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90)
>>         at
>> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3347)
>>         at
>> org.apache.hadoop.hbase.client.HBaseAdmin.listTableDescriptorsByNamespace(HBaseAdmin.java:2281)
>>         at
>> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.__jamon_innerUnit__catalogTables(MasterStatusTmplImpl.java:465)
>>         at
>> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.renderNoFlush(MasterStatusTmplImpl.java:274)
>>         at
>> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.renderNoFlush(MasterStatusTmpl.java:386)
>>         at
>> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.render(MasterStatusTmpl.java:376)
>>         at
>> org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:95)
>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>>         at
>> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>>         at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
>>         at
>> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
>>         at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>         at
>> org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1122)
>>         at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>         at
>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>>         at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>         at java.io.FilterInputStream.read(FilterInputStream.java:133)
>>         at
>> org.apache.hadoop.hbase.ipc.RpcClient$Connection$PingInputStream.read(RpcClient.java:558)
>>         at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
>>         at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
>>         at java.io.DataInputStream.readInt(DataInputStream.java:387)
>>         at
>> org.apache.hadoop.hbase.ipc.RpcClient$Connection.readResponse(RpcClient.java:1076)
>>         at
>> org.apache.hadoop.hbase.ipc.RpcClient$Connection.run(RpcClient.java:727)
>>
>> Thanks
>>
>> > On 03 Sep 2015, at 15:50, Ted Yu <yu...@gmail.com> wrote:
>> >
>> > Can you log into the master node and check master log ?
>> >
>> > Pastebin snippet of master log if you need more help.
>> >
>> > Cheers
>> >
>> >
>> >> On Sep 3, 2015, at 6:24 AM, Akmal Abbasov <ak...@icloud.com>
>> wrote:
>> >>
>> >> Hi,
>> >> I’m having problems with accessing HBase master webui. when I try to
>> access it, it returns redirect page /master-status, and then it is freezes.
>> >> I’ve tried to wait for a long time, but still it didn’t work.
>> >> I am using hbase hbase-0.98.7-hadoop2, hadoop hadoop-2.5.1
>> >> Any suggestions?
>> >> Thanks.
>>
>>
>
>

Re: HBase master WebUI is not responding

Posted by Ted Yu <yu...@gmail.com>.
Can you do the following ?

run hbck command to see if any inconsistency is reported
check region server log on test-rs3 to see if hbase:meta is successfully
opened

Cheers

On Fri, Sep 4, 2015 at 8:20 AM, Akmal Abbasov <ak...@icloud.com>
wrote:

> Hi Ted,
>
> No, I’ve restarted master node and now there is no exceptions, but still
> webui is not responding
> 2015-09-04 14:54:57,926 DEBUG [FifoRpcScheduler.handler1-thread-26]
> master.AssignmentManager: based on AM, current
> region=hbase:meta,,1.1588230740 is on server=test-rs3,60020,1441378286999
> server being checked: test-rs5,60020,1441261824016
> What else could be wrong?
>
> Thanks.
>
>
> On 03 Sep 2015, at 16:20, Ted Yu <yu...@gmail.com> wrote:
>
> Did the exception lead to hbase:meta assignment failure ?
>
> Please check region server log on 10.10.8.53 <http://10.10.8.53:60000/>
>
> On Thu, Sep 3, 2015 at 7:15 AM, Akmal Abbasov <ak...@icloud.com>
> wrote:
>
>> Hi Ted,
>> I’m getting a lot of
>> 2015-09-03 14:08:20,651 WARN  [1865504699@qtp-838397579-13]
>> client.HConnectionManager$HConnectionImplementation: Checking master
>> connection
>> com.google.protobuf.ServiceException: java.net.SocketTimeoutException:
>> Call to test-m/10.10.8.53:60000 failed because
>> java.net.SocketTimeoutException: 60000 millis timeout while waiting for
>> channel to be ready for read. ch :
>> java.nio.channels.SocketChannel[connected local=/10.10.8.53:41243
>> remote=test-m/10.10.8.53:60000]
>>         at
>> org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1678)
>>         at
>> org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1719)
>>         at
>> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561)
>>         at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceState.isMasterRunning(HConnectionManager.java:1508)
>>         at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isKeepAliveMasterConnectedAndRunning(HConnectionManager.java:2146)
>>         at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1859)
>>         at
>> org.apache.hadoop.hbase.client.HBaseAdmin$MasterCallable.prepare(HBaseAdmin.java:3320)
>>         at
>> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:113)
>>         at
>> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90)
>>         at
>> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3347)
>>         at
>> org.apache.hadoop.hbase.client.HBaseAdmin.listTableDescriptorsByNamespace(HBaseAdmin.java:2281)
>>         at
>> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.__jamon_innerUnit__catalogTables(MasterStatusTmplImpl.java:465)
>>         at
>> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.renderNoFlush(MasterStatusTmplImpl.java:274)
>>         at
>> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.renderNoFlush(MasterStatusTmpl.java:386)
>>         at
>> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.render(MasterStatusTmpl.java:376)
>>         at
>> org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:95)
>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>>         at
>> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>>         at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
>>         at
>> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
>>         at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>         at
>> org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1122)
>>         at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>         at
>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>>         at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>         at java.io.FilterInputStream.read(FilterInputStream.java:133)
>>         at
>> org.apache.hadoop.hbase.ipc.RpcClient$Connection$PingInputStream.read(RpcClient.java:558)
>>         at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
>>         at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
>>         at java.io.DataInputStream.readInt(DataInputStream.java:387)
>>         at
>> org.apache.hadoop.hbase.ipc.RpcClient$Connection.readResponse(RpcClient.java:1076)
>>         at
>> org.apache.hadoop.hbase.ipc.RpcClient$Connection.run(RpcClient.java:727)
>>
>> Thanks
>>
>> > On 03 Sep 2015, at 15:50, Ted Yu <yu...@gmail.com> wrote:
>> >
>> > Can you log into the master node and check master log ?
>> >
>> > Pastebin snippet of master log if you need more help.
>> >
>> > Cheers
>> >
>> >
>> >> On Sep 3, 2015, at 6:24 AM, Akmal Abbasov <ak...@icloud.com>
>> wrote:
>> >>
>> >> Hi,
>> >> I’m having problems with accessing HBase master webui. when I try to
>> access it, it returns redirect page /master-status, and then it is freezes.
>> >> I’ve tried to wait for a long time, but still it didn’t work.
>> >> I am using hbase hbase-0.98.7-hadoop2, hadoop hadoop-2.5.1
>> >> Any suggestions?
>> >> Thanks.
>>
>>
>
>

Re: HBase master WebUI is not responding

Posted by Ted Yu <yu...@gmail.com>.
Can you do the following ?

run hbck command to see if any inconsistency is reported
check region server log on test-rs3 to see if hbase:meta is successfully
opened

Cheers

On Fri, Sep 4, 2015 at 8:20 AM, Akmal Abbasov <ak...@icloud.com>
wrote:

> Hi Ted,
>
> No, I’ve restarted master node and now there is no exceptions, but still
> webui is not responding
> 2015-09-04 14:54:57,926 DEBUG [FifoRpcScheduler.handler1-thread-26]
> master.AssignmentManager: based on AM, current
> region=hbase:meta,,1.1588230740 is on server=test-rs3,60020,1441378286999
> server being checked: test-rs5,60020,1441261824016
> What else could be wrong?
>
> Thanks.
>
>
> On 03 Sep 2015, at 16:20, Ted Yu <yu...@gmail.com> wrote:
>
> Did the exception lead to hbase:meta assignment failure ?
>
> Please check region server log on 10.10.8.53 <http://10.10.8.53:60000/>
>
> On Thu, Sep 3, 2015 at 7:15 AM, Akmal Abbasov <ak...@icloud.com>
> wrote:
>
>> Hi Ted,
>> I’m getting a lot of
>> 2015-09-03 14:08:20,651 WARN  [1865504699@qtp-838397579-13]
>> client.HConnectionManager$HConnectionImplementation: Checking master
>> connection
>> com.google.protobuf.ServiceException: java.net.SocketTimeoutException:
>> Call to test-m/10.10.8.53:60000 failed because
>> java.net.SocketTimeoutException: 60000 millis timeout while waiting for
>> channel to be ready for read. ch :
>> java.nio.channels.SocketChannel[connected local=/10.10.8.53:41243
>> remote=test-m/10.10.8.53:60000]
>>         at
>> org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1678)
>>         at
>> org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1719)
>>         at
>> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561)
>>         at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceState.isMasterRunning(HConnectionManager.java:1508)
>>         at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isKeepAliveMasterConnectedAndRunning(HConnectionManager.java:2146)
>>         at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1859)
>>         at
>> org.apache.hadoop.hbase.client.HBaseAdmin$MasterCallable.prepare(HBaseAdmin.java:3320)
>>         at
>> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:113)
>>         at
>> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90)
>>         at
>> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3347)
>>         at
>> org.apache.hadoop.hbase.client.HBaseAdmin.listTableDescriptorsByNamespace(HBaseAdmin.java:2281)
>>         at
>> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.__jamon_innerUnit__catalogTables(MasterStatusTmplImpl.java:465)
>>         at
>> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.renderNoFlush(MasterStatusTmplImpl.java:274)
>>         at
>> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.renderNoFlush(MasterStatusTmpl.java:386)
>>         at
>> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.render(MasterStatusTmpl.java:376)
>>         at
>> org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:95)
>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>>         at
>> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>>         at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
>>         at
>> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
>>         at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>         at
>> org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1122)
>>         at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>         at
>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>>         at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>         at java.io.FilterInputStream.read(FilterInputStream.java:133)
>>         at
>> org.apache.hadoop.hbase.ipc.RpcClient$Connection$PingInputStream.read(RpcClient.java:558)
>>         at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
>>         at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
>>         at java.io.DataInputStream.readInt(DataInputStream.java:387)
>>         at
>> org.apache.hadoop.hbase.ipc.RpcClient$Connection.readResponse(RpcClient.java:1076)
>>         at
>> org.apache.hadoop.hbase.ipc.RpcClient$Connection.run(RpcClient.java:727)
>>
>> Thanks
>>
>> > On 03 Sep 2015, at 15:50, Ted Yu <yu...@gmail.com> wrote:
>> >
>> > Can you log into the master node and check master log ?
>> >
>> > Pastebin snippet of master log if you need more help.
>> >
>> > Cheers
>> >
>> >
>> >> On Sep 3, 2015, at 6:24 AM, Akmal Abbasov <ak...@icloud.com>
>> wrote:
>> >>
>> >> Hi,
>> >> I’m having problems with accessing HBase master webui. when I try to
>> access it, it returns redirect page /master-status, and then it is freezes.
>> >> I’ve tried to wait for a long time, but still it didn’t work.
>> >> I am using hbase hbase-0.98.7-hadoop2, hadoop hadoop-2.5.1
>> >> Any suggestions?
>> >> Thanks.
>>
>>
>
>

Re: HBase master WebUI is not responding

Posted by Akmal Abbasov <ak...@icloud.com>.
Hi Ted, 

No, I’ve restarted master node and now there is no exceptions, but still webui is not responding
2015-09-04 14:54:57,926 DEBUG [FifoRpcScheduler.handler1-thread-26] master.AssignmentManager: based on AM, current region=hbase:meta,,1.1588230740 is on server=test-rs3,60020,1441378286999 server being checked: test-rs5,60020,1441261824016
What else could be wrong?

Thanks.


> On 03 Sep 2015, at 16:20, Ted Yu <yu...@gmail.com> wrote:
> 
> Did the exception lead to hbase:meta assignment failure ?
> 
> Please check region server log on 10.10.8.53 <http://10.10.8.53:60000/>
> 
> On Thu, Sep 3, 2015 at 7:15 AM, Akmal Abbasov <akmal.abbasov@icloud.com <ma...@icloud.com>> wrote:
> Hi Ted,
> I’m getting a lot of
> 2015-09-03 14:08:20,651 WARN  [1865504699@qtp-838397579-13] client.HConnectionManager$HConnectionImplementation: Checking master connection
> com.google.protobuf.ServiceException: java.net.SocketTimeoutException: Call to test-m/10.10.8.53:60000 <http://10.10.8.53:60000/> failed because java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/10.10.8.53:41243 <http://10.10.8.53:41243/> remote=test-m/10.10.8.53:60000 <http://10.10.8.53:60000/>]
>         at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1678)
>         at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1719)
>         at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561)
>         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceState.isMasterRunning(HConnectionManager.java:1508)
>         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isKeepAliveMasterConnectedAndRunning(HConnectionManager.java:2146)
>         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1859)
>         at org.apache.hadoop.hbase.client.HBaseAdmin$MasterCallable.prepare(HBaseAdmin.java:3320)
>         at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:113)
>         at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90)
>         at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3347)
>         at org.apache.hadoop.hbase.client.HBaseAdmin.listTableDescriptorsByNamespace(HBaseAdmin.java:2281)
>         at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.__jamon_innerUnit__catalogTables(MasterStatusTmplImpl.java:465)
>         at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.renderNoFlush(MasterStatusTmplImpl.java:274)
>         at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.renderNoFlush(MasterStatusTmpl.java:386)
>         at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.render(MasterStatusTmpl.java:376)
>         at org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:95)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>         at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
>         at org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
>         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1122)
>         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at java.io.FilterInputStream.read(FilterInputStream.java:133)
>         at org.apache.hadoop.hbase.ipc.RpcClient$Connection$PingInputStream.read(RpcClient.java:558)
>         at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
>         at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
>         at java.io.DataInputStream.readInt(DataInputStream.java:387)
>         at org.apache.hadoop.hbase.ipc.RpcClient$Connection.readResponse(RpcClient.java:1076)
>         at org.apache.hadoop.hbase.ipc.RpcClient$Connection.run(RpcClient.java:727)
> 
> Thanks
> 
> > On 03 Sep 2015, at 15:50, Ted Yu <yuzhihong@gmail.com <ma...@gmail.com>> wrote:
> >
> > Can you log into the master node and check master log ?
> >
> > Pastebin snippet of master log if you need more help.
> >
> > Cheers
> >
> >
> >> On Sep 3, 2015, at 6:24 AM, Akmal Abbasov <akmal.abbasov@icloud.com <ma...@icloud.com>> wrote:
> >>
> >> Hi,
> >> I’m having problems with accessing HBase master webui. when I try to access it, it returns redirect page /master-status, and then it is freezes.
> >> I’ve tried to wait for a long time, but still it didn’t work.
> >> I am using hbase hbase-0.98.7-hadoop2, hadoop hadoop-2.5.1
> >> Any suggestions?
> >> Thanks.
> 
> 


Re: HBase master WebUI is not responding

Posted by Akmal Abbasov <ak...@icloud.com>.
Hi Ted, 

No, I’ve restarted master node and now there is no exceptions, but still webui is not responding
2015-09-04 14:54:57,926 DEBUG [FifoRpcScheduler.handler1-thread-26] master.AssignmentManager: based on AM, current region=hbase:meta,,1.1588230740 is on server=test-rs3,60020,1441378286999 server being checked: test-rs5,60020,1441261824016
What else could be wrong?

Thanks.


> On 03 Sep 2015, at 16:20, Ted Yu <yu...@gmail.com> wrote:
> 
> Did the exception lead to hbase:meta assignment failure ?
> 
> Please check region server log on 10.10.8.53 <http://10.10.8.53:60000/>
> 
> On Thu, Sep 3, 2015 at 7:15 AM, Akmal Abbasov <akmal.abbasov@icloud.com <ma...@icloud.com>> wrote:
> Hi Ted,
> I’m getting a lot of
> 2015-09-03 14:08:20,651 WARN  [1865504699@qtp-838397579-13] client.HConnectionManager$HConnectionImplementation: Checking master connection
> com.google.protobuf.ServiceException: java.net.SocketTimeoutException: Call to test-m/10.10.8.53:60000 <http://10.10.8.53:60000/> failed because java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/10.10.8.53:41243 <http://10.10.8.53:41243/> remote=test-m/10.10.8.53:60000 <http://10.10.8.53:60000/>]
>         at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1678)
>         at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1719)
>         at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561)
>         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceState.isMasterRunning(HConnectionManager.java:1508)
>         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isKeepAliveMasterConnectedAndRunning(HConnectionManager.java:2146)
>         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1859)
>         at org.apache.hadoop.hbase.client.HBaseAdmin$MasterCallable.prepare(HBaseAdmin.java:3320)
>         at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:113)
>         at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90)
>         at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3347)
>         at org.apache.hadoop.hbase.client.HBaseAdmin.listTableDescriptorsByNamespace(HBaseAdmin.java:2281)
>         at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.__jamon_innerUnit__catalogTables(MasterStatusTmplImpl.java:465)
>         at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.renderNoFlush(MasterStatusTmplImpl.java:274)
>         at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.renderNoFlush(MasterStatusTmpl.java:386)
>         at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.render(MasterStatusTmpl.java:376)
>         at org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:95)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>         at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
>         at org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
>         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1122)
>         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at java.io.FilterInputStream.read(FilterInputStream.java:133)
>         at org.apache.hadoop.hbase.ipc.RpcClient$Connection$PingInputStream.read(RpcClient.java:558)
>         at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
>         at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
>         at java.io.DataInputStream.readInt(DataInputStream.java:387)
>         at org.apache.hadoop.hbase.ipc.RpcClient$Connection.readResponse(RpcClient.java:1076)
>         at org.apache.hadoop.hbase.ipc.RpcClient$Connection.run(RpcClient.java:727)
> 
> Thanks
> 
> > On 03 Sep 2015, at 15:50, Ted Yu <yuzhihong@gmail.com <ma...@gmail.com>> wrote:
> >
> > Can you log into the master node and check master log ?
> >
> > Pastebin snippet of master log if you need more help.
> >
> > Cheers
> >
> >
> >> On Sep 3, 2015, at 6:24 AM, Akmal Abbasov <akmal.abbasov@icloud.com <ma...@icloud.com>> wrote:
> >>
> >> Hi,
> >> I’m having problems with accessing HBase master webui. when I try to access it, it returns redirect page /master-status, and then it is freezes.
> >> I’ve tried to wait for a long time, but still it didn’t work.
> >> I am using hbase hbase-0.98.7-hadoop2, hadoop hadoop-2.5.1
> >> Any suggestions?
> >> Thanks.
> 
> 


Re: HBase master WebUI is not responding

Posted by Akmal Abbasov <ak...@icloud.com>.
Hi Ted, 

No, I’ve restarted master node and now there is no exceptions, but still webui is not responding
2015-09-04 14:54:57,926 DEBUG [FifoRpcScheduler.handler1-thread-26] master.AssignmentManager: based on AM, current region=hbase:meta,,1.1588230740 is on server=test-rs3,60020,1441378286999 server being checked: test-rs5,60020,1441261824016
What else could be wrong?

Thanks.


> On 03 Sep 2015, at 16:20, Ted Yu <yu...@gmail.com> wrote:
> 
> Did the exception lead to hbase:meta assignment failure ?
> 
> Please check region server log on 10.10.8.53 <http://10.10.8.53:60000/>
> 
> On Thu, Sep 3, 2015 at 7:15 AM, Akmal Abbasov <akmal.abbasov@icloud.com <ma...@icloud.com>> wrote:
> Hi Ted,
> I’m getting a lot of
> 2015-09-03 14:08:20,651 WARN  [1865504699@qtp-838397579-13] client.HConnectionManager$HConnectionImplementation: Checking master connection
> com.google.protobuf.ServiceException: java.net.SocketTimeoutException: Call to test-m/10.10.8.53:60000 <http://10.10.8.53:60000/> failed because java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/10.10.8.53:41243 <http://10.10.8.53:41243/> remote=test-m/10.10.8.53:60000 <http://10.10.8.53:60000/>]
>         at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1678)
>         at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1719)
>         at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561)
>         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceState.isMasterRunning(HConnectionManager.java:1508)
>         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isKeepAliveMasterConnectedAndRunning(HConnectionManager.java:2146)
>         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1859)
>         at org.apache.hadoop.hbase.client.HBaseAdmin$MasterCallable.prepare(HBaseAdmin.java:3320)
>         at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:113)
>         at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90)
>         at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3347)
>         at org.apache.hadoop.hbase.client.HBaseAdmin.listTableDescriptorsByNamespace(HBaseAdmin.java:2281)
>         at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.__jamon_innerUnit__catalogTables(MasterStatusTmplImpl.java:465)
>         at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.renderNoFlush(MasterStatusTmplImpl.java:274)
>         at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.renderNoFlush(MasterStatusTmpl.java:386)
>         at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.render(MasterStatusTmpl.java:376)
>         at org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:95)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>         at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
>         at org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
>         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1122)
>         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at java.io.FilterInputStream.read(FilterInputStream.java:133)
>         at org.apache.hadoop.hbase.ipc.RpcClient$Connection$PingInputStream.read(RpcClient.java:558)
>         at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
>         at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
>         at java.io.DataInputStream.readInt(DataInputStream.java:387)
>         at org.apache.hadoop.hbase.ipc.RpcClient$Connection.readResponse(RpcClient.java:1076)
>         at org.apache.hadoop.hbase.ipc.RpcClient$Connection.run(RpcClient.java:727)
> 
> Thanks
> 
> > On 03 Sep 2015, at 15:50, Ted Yu <yuzhihong@gmail.com <ma...@gmail.com>> wrote:
> >
> > Can you log into the master node and check master log ?
> >
> > Pastebin snippet of master log if you need more help.
> >
> > Cheers
> >
> >
> >> On Sep 3, 2015, at 6:24 AM, Akmal Abbasov <akmal.abbasov@icloud.com <ma...@icloud.com>> wrote:
> >>
> >> Hi,
> >> I’m having problems with accessing HBase master webui. when I try to access it, it returns redirect page /master-status, and then it is freezes.
> >> I’ve tried to wait for a long time, but still it didn’t work.
> >> I am using hbase hbase-0.98.7-hadoop2, hadoop hadoop-2.5.1
> >> Any suggestions?
> >> Thanks.
> 
> 


Re: HBase master WebUI is not responding

Posted by Akmal Abbasov <ak...@icloud.com>.
Hi Ted, 

No, I’ve restarted master node and now there is no exceptions, but still webui is not responding
2015-09-04 14:54:57,926 DEBUG [FifoRpcScheduler.handler1-thread-26] master.AssignmentManager: based on AM, current region=hbase:meta,,1.1588230740 is on server=test-rs3,60020,1441378286999 server being checked: test-rs5,60020,1441261824016
What else could be wrong?

Thanks.


> On 03 Sep 2015, at 16:20, Ted Yu <yu...@gmail.com> wrote:
> 
> Did the exception lead to hbase:meta assignment failure ?
> 
> Please check region server log on 10.10.8.53 <http://10.10.8.53:60000/>
> 
> On Thu, Sep 3, 2015 at 7:15 AM, Akmal Abbasov <akmal.abbasov@icloud.com <ma...@icloud.com>> wrote:
> Hi Ted,
> I’m getting a lot of
> 2015-09-03 14:08:20,651 WARN  [1865504699@qtp-838397579-13] client.HConnectionManager$HConnectionImplementation: Checking master connection
> com.google.protobuf.ServiceException: java.net.SocketTimeoutException: Call to test-m/10.10.8.53:60000 <http://10.10.8.53:60000/> failed because java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/10.10.8.53:41243 <http://10.10.8.53:41243/> remote=test-m/10.10.8.53:60000 <http://10.10.8.53:60000/>]
>         at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1678)
>         at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1719)
>         at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561)
>         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceState.isMasterRunning(HConnectionManager.java:1508)
>         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isKeepAliveMasterConnectedAndRunning(HConnectionManager.java:2146)
>         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1859)
>         at org.apache.hadoop.hbase.client.HBaseAdmin$MasterCallable.prepare(HBaseAdmin.java:3320)
>         at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:113)
>         at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90)
>         at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3347)
>         at org.apache.hadoop.hbase.client.HBaseAdmin.listTableDescriptorsByNamespace(HBaseAdmin.java:2281)
>         at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.__jamon_innerUnit__catalogTables(MasterStatusTmplImpl.java:465)
>         at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.renderNoFlush(MasterStatusTmplImpl.java:274)
>         at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.renderNoFlush(MasterStatusTmpl.java:386)
>         at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.render(MasterStatusTmpl.java:376)
>         at org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:95)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>         at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
>         at org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
>         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1122)
>         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at java.io.FilterInputStream.read(FilterInputStream.java:133)
>         at org.apache.hadoop.hbase.ipc.RpcClient$Connection$PingInputStream.read(RpcClient.java:558)
>         at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
>         at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
>         at java.io.DataInputStream.readInt(DataInputStream.java:387)
>         at org.apache.hadoop.hbase.ipc.RpcClient$Connection.readResponse(RpcClient.java:1076)
>         at org.apache.hadoop.hbase.ipc.RpcClient$Connection.run(RpcClient.java:727)
> 
> Thanks
> 
> > On 03 Sep 2015, at 15:50, Ted Yu <yuzhihong@gmail.com <ma...@gmail.com>> wrote:
> >
> > Can you log into the master node and check master log ?
> >
> > Pastebin snippet of master log if you need more help.
> >
> > Cheers
> >
> >
> >> On Sep 3, 2015, at 6:24 AM, Akmal Abbasov <akmal.abbasov@icloud.com <ma...@icloud.com>> wrote:
> >>
> >> Hi,
> >> I’m having problems with accessing HBase master webui. when I try to access it, it returns redirect page /master-status, and then it is freezes.
> >> I’ve tried to wait for a long time, but still it didn’t work.
> >> I am using hbase hbase-0.98.7-hadoop2, hadoop hadoop-2.5.1
> >> Any suggestions?
> >> Thanks.
> 
> 


Re: HBase master WebUI is not responding

Posted by Ted Yu <yu...@gmail.com>.
Did the exception lead to hbase:meta assignment failure ?

Please check region server log on 10.10.8.53 <http://10.10.8.53:60000/>

On Thu, Sep 3, 2015 at 7:15 AM, Akmal Abbasov <ak...@icloud.com>
wrote:

> Hi Ted,
> I’m getting a lot of
> 2015-09-03 14:08:20,651 WARN  [1865504699@qtp-838397579-13]
> client.HConnectionManager$HConnectionImplementation: Checking master
> connection
> com.google.protobuf.ServiceException: java.net.SocketTimeoutException:
> Call to test-m/10.10.8.53:60000 failed because
> java.net.SocketTimeoutException: 60000 millis timeout while waiting for
> channel to be ready for read. ch :
> java.nio.channels.SocketChannel[connected local=/10.10.8.53:41243
> remote=test-m/10.10.8.53:60000]
>         at
> org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1678)
>         at
> org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1719)
>         at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561)
>         at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceState.isMasterRunning(HConnectionManager.java:1508)
>         at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isKeepAliveMasterConnectedAndRunning(HConnectionManager.java:2146)
>         at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1859)
>         at
> org.apache.hadoop.hbase.client.HBaseAdmin$MasterCallable.prepare(HBaseAdmin.java:3320)
>         at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:113)
>         at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90)
>         at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3347)
>         at
> org.apache.hadoop.hbase.client.HBaseAdmin.listTableDescriptorsByNamespace(HBaseAdmin.java:2281)
>         at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.__jamon_innerUnit__catalogTables(MasterStatusTmplImpl.java:465)
>         at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.renderNoFlush(MasterStatusTmplImpl.java:274)
>         at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.renderNoFlush(MasterStatusTmpl.java:386)
>         at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.render(MasterStatusTmpl.java:376)
>         at
> org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:95)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>         at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>         at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
>         at
> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
>         at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at
> org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1122)
>         at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at
> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>         at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at java.io.FilterInputStream.read(FilterInputStream.java:133)
>         at
> org.apache.hadoop.hbase.ipc.RpcClient$Connection$PingInputStream.read(RpcClient.java:558)
>         at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
>         at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
>         at java.io.DataInputStream.readInt(DataInputStream.java:387)
>         at
> org.apache.hadoop.hbase.ipc.RpcClient$Connection.readResponse(RpcClient.java:1076)
>         at
> org.apache.hadoop.hbase.ipc.RpcClient$Connection.run(RpcClient.java:727)
>
> Thanks
>
> > On 03 Sep 2015, at 15:50, Ted Yu <yu...@gmail.com> wrote:
> >
> > Can you log into the master node and check master log ?
> >
> > Pastebin snippet of master log if you need more help.
> >
> > Cheers
> >
> >
> >> On Sep 3, 2015, at 6:24 AM, Akmal Abbasov <ak...@icloud.com>
> wrote:
> >>
> >> Hi,
> >> I’m having problems with accessing HBase master webui. when I try to
> access it, it returns redirect page /master-status, and then it is freezes.
> >> I’ve tried to wait for a long time, but still it didn’t work.
> >> I am using hbase hbase-0.98.7-hadoop2, hadoop hadoop-2.5.1
> >> Any suggestions?
> >> Thanks.
>
>

Re: HBase master WebUI is not responding

Posted by Ted Yu <yu...@gmail.com>.
Did the exception lead to hbase:meta assignment failure ?

Please check region server log on 10.10.8.53 <http://10.10.8.53:60000/>

On Thu, Sep 3, 2015 at 7:15 AM, Akmal Abbasov <ak...@icloud.com>
wrote:

> Hi Ted,
> I’m getting a lot of
> 2015-09-03 14:08:20,651 WARN  [1865504699@qtp-838397579-13]
> client.HConnectionManager$HConnectionImplementation: Checking master
> connection
> com.google.protobuf.ServiceException: java.net.SocketTimeoutException:
> Call to test-m/10.10.8.53:60000 failed because
> java.net.SocketTimeoutException: 60000 millis timeout while waiting for
> channel to be ready for read. ch :
> java.nio.channels.SocketChannel[connected local=/10.10.8.53:41243
> remote=test-m/10.10.8.53:60000]
>         at
> org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1678)
>         at
> org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1719)
>         at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561)
>         at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceState.isMasterRunning(HConnectionManager.java:1508)
>         at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isKeepAliveMasterConnectedAndRunning(HConnectionManager.java:2146)
>         at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1859)
>         at
> org.apache.hadoop.hbase.client.HBaseAdmin$MasterCallable.prepare(HBaseAdmin.java:3320)
>         at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:113)
>         at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90)
>         at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3347)
>         at
> org.apache.hadoop.hbase.client.HBaseAdmin.listTableDescriptorsByNamespace(HBaseAdmin.java:2281)
>         at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.__jamon_innerUnit__catalogTables(MasterStatusTmplImpl.java:465)
>         at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.renderNoFlush(MasterStatusTmplImpl.java:274)
>         at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.renderNoFlush(MasterStatusTmpl.java:386)
>         at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.render(MasterStatusTmpl.java:376)
>         at
> org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:95)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>         at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>         at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
>         at
> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
>         at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at
> org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1122)
>         at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at
> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>         at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at java.io.FilterInputStream.read(FilterInputStream.java:133)
>         at
> org.apache.hadoop.hbase.ipc.RpcClient$Connection$PingInputStream.read(RpcClient.java:558)
>         at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
>         at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
>         at java.io.DataInputStream.readInt(DataInputStream.java:387)
>         at
> org.apache.hadoop.hbase.ipc.RpcClient$Connection.readResponse(RpcClient.java:1076)
>         at
> org.apache.hadoop.hbase.ipc.RpcClient$Connection.run(RpcClient.java:727)
>
> Thanks
>
> > On 03 Sep 2015, at 15:50, Ted Yu <yu...@gmail.com> wrote:
> >
> > Can you log into the master node and check master log ?
> >
> > Pastebin snippet of master log if you need more help.
> >
> > Cheers
> >
> >
> >> On Sep 3, 2015, at 6:24 AM, Akmal Abbasov <ak...@icloud.com>
> wrote:
> >>
> >> Hi,
> >> I’m having problems with accessing HBase master webui. when I try to
> access it, it returns redirect page /master-status, and then it is freezes.
> >> I’ve tried to wait for a long time, but still it didn’t work.
> >> I am using hbase hbase-0.98.7-hadoop2, hadoop hadoop-2.5.1
> >> Any suggestions?
> >> Thanks.
>
>

Re: HBase master WebUI is not responding

Posted by Ted Yu <yu...@gmail.com>.
Did the exception lead to hbase:meta assignment failure ?

Please check region server log on 10.10.8.53 <http://10.10.8.53:60000/>

On Thu, Sep 3, 2015 at 7:15 AM, Akmal Abbasov <ak...@icloud.com>
wrote:

> Hi Ted,
> I’m getting a lot of
> 2015-09-03 14:08:20,651 WARN  [1865504699@qtp-838397579-13]
> client.HConnectionManager$HConnectionImplementation: Checking master
> connection
> com.google.protobuf.ServiceException: java.net.SocketTimeoutException:
> Call to test-m/10.10.8.53:60000 failed because
> java.net.SocketTimeoutException: 60000 millis timeout while waiting for
> channel to be ready for read. ch :
> java.nio.channels.SocketChannel[connected local=/10.10.8.53:41243
> remote=test-m/10.10.8.53:60000]
>         at
> org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1678)
>         at
> org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1719)
>         at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561)
>         at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceState.isMasterRunning(HConnectionManager.java:1508)
>         at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isKeepAliveMasterConnectedAndRunning(HConnectionManager.java:2146)
>         at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1859)
>         at
> org.apache.hadoop.hbase.client.HBaseAdmin$MasterCallable.prepare(HBaseAdmin.java:3320)
>         at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:113)
>         at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90)
>         at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3347)
>         at
> org.apache.hadoop.hbase.client.HBaseAdmin.listTableDescriptorsByNamespace(HBaseAdmin.java:2281)
>         at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.__jamon_innerUnit__catalogTables(MasterStatusTmplImpl.java:465)
>         at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.renderNoFlush(MasterStatusTmplImpl.java:274)
>         at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.renderNoFlush(MasterStatusTmpl.java:386)
>         at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.render(MasterStatusTmpl.java:376)
>         at
> org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:95)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>         at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>         at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
>         at
> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
>         at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at
> org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1122)
>         at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at
> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>         at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at java.io.FilterInputStream.read(FilterInputStream.java:133)
>         at
> org.apache.hadoop.hbase.ipc.RpcClient$Connection$PingInputStream.read(RpcClient.java:558)
>         at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
>         at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
>         at java.io.DataInputStream.readInt(DataInputStream.java:387)
>         at
> org.apache.hadoop.hbase.ipc.RpcClient$Connection.readResponse(RpcClient.java:1076)
>         at
> org.apache.hadoop.hbase.ipc.RpcClient$Connection.run(RpcClient.java:727)
>
> Thanks
>
> > On 03 Sep 2015, at 15:50, Ted Yu <yu...@gmail.com> wrote:
> >
> > Can you log into the master node and check master log ?
> >
> > Pastebin snippet of master log if you need more help.
> >
> > Cheers
> >
> >
> >> On Sep 3, 2015, at 6:24 AM, Akmal Abbasov <ak...@icloud.com>
> wrote:
> >>
> >> Hi,
> >> I’m having problems with accessing HBase master webui. when I try to
> access it, it returns redirect page /master-status, and then it is freezes.
> >> I’ve tried to wait for a long time, but still it didn’t work.
> >> I am using hbase hbase-0.98.7-hadoop2, hadoop hadoop-2.5.1
> >> Any suggestions?
> >> Thanks.
>
>

Re: HBase master WebUI is not responding

Posted by Ted Yu <yu...@gmail.com>.
Did the exception lead to hbase:meta assignment failure ?

Please check region server log on 10.10.8.53 <http://10.10.8.53:60000/>

On Thu, Sep 3, 2015 at 7:15 AM, Akmal Abbasov <ak...@icloud.com>
wrote:

> Hi Ted,
> I’m getting a lot of
> 2015-09-03 14:08:20,651 WARN  [1865504699@qtp-838397579-13]
> client.HConnectionManager$HConnectionImplementation: Checking master
> connection
> com.google.protobuf.ServiceException: java.net.SocketTimeoutException:
> Call to test-m/10.10.8.53:60000 failed because
> java.net.SocketTimeoutException: 60000 millis timeout while waiting for
> channel to be ready for read. ch :
> java.nio.channels.SocketChannel[connected local=/10.10.8.53:41243
> remote=test-m/10.10.8.53:60000]
>         at
> org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1678)
>         at
> org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1719)
>         at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561)
>         at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceState.isMasterRunning(HConnectionManager.java:1508)
>         at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isKeepAliveMasterConnectedAndRunning(HConnectionManager.java:2146)
>         at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1859)
>         at
> org.apache.hadoop.hbase.client.HBaseAdmin$MasterCallable.prepare(HBaseAdmin.java:3320)
>         at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:113)
>         at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90)
>         at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3347)
>         at
> org.apache.hadoop.hbase.client.HBaseAdmin.listTableDescriptorsByNamespace(HBaseAdmin.java:2281)
>         at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.__jamon_innerUnit__catalogTables(MasterStatusTmplImpl.java:465)
>         at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.renderNoFlush(MasterStatusTmplImpl.java:274)
>         at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.renderNoFlush(MasterStatusTmpl.java:386)
>         at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.render(MasterStatusTmpl.java:376)
>         at
> org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:95)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>         at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>         at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
>         at
> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
>         at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at
> org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1122)
>         at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at
> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>         at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at java.io.FilterInputStream.read(FilterInputStream.java:133)
>         at
> org.apache.hadoop.hbase.ipc.RpcClient$Connection$PingInputStream.read(RpcClient.java:558)
>         at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
>         at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
>         at java.io.DataInputStream.readInt(DataInputStream.java:387)
>         at
> org.apache.hadoop.hbase.ipc.RpcClient$Connection.readResponse(RpcClient.java:1076)
>         at
> org.apache.hadoop.hbase.ipc.RpcClient$Connection.run(RpcClient.java:727)
>
> Thanks
>
> > On 03 Sep 2015, at 15:50, Ted Yu <yu...@gmail.com> wrote:
> >
> > Can you log into the master node and check master log ?
> >
> > Pastebin snippet of master log if you need more help.
> >
> > Cheers
> >
> >
> >> On Sep 3, 2015, at 6:24 AM, Akmal Abbasov <ak...@icloud.com>
> wrote:
> >>
> >> Hi,
> >> I’m having problems with accessing HBase master webui. when I try to
> access it, it returns redirect page /master-status, and then it is freezes.
> >> I’ve tried to wait for a long time, but still it didn’t work.
> >> I am using hbase hbase-0.98.7-hadoop2, hadoop hadoop-2.5.1
> >> Any suggestions?
> >> Thanks.
>
>

Re: HBase master WebUI is not responding

Posted by Akmal Abbasov <ak...@icloud.com>.
Hi Ted,
I’m getting a lot of 
2015-09-03 14:08:20,651 WARN  [1865504699@qtp-838397579-13] client.HConnectionManager$HConnectionImplementation: Checking master connection
com.google.protobuf.ServiceException: java.net.SocketTimeoutException: Call to test-m/10.10.8.53:60000 failed because java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/10.10.8.53:41243 remote=test-m/10.10.8.53:60000]
        at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1678)
        at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1719)
        at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceState.isMasterRunning(HConnectionManager.java:1508)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isKeepAliveMasterConnectedAndRunning(HConnectionManager.java:2146)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1859)
        at org.apache.hadoop.hbase.client.HBaseAdmin$MasterCallable.prepare(HBaseAdmin.java:3320)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:113)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90)
        at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3347)
        at org.apache.hadoop.hbase.client.HBaseAdmin.listTableDescriptorsByNamespace(HBaseAdmin.java:2281)
        at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.__jamon_innerUnit__catalogTables(MasterStatusTmplImpl.java:465)
        at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.renderNoFlush(MasterStatusTmplImpl.java:274)
        at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.renderNoFlush(MasterStatusTmpl.java:386)
        at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.render(MasterStatusTmpl.java:376)
        at org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:95)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
        at org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
        at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1122)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
        at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
        at java.io.FilterInputStream.read(FilterInputStream.java:133)
        at org.apache.hadoop.hbase.ipc.RpcClient$Connection$PingInputStream.read(RpcClient.java:558)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
        at java.io.DataInputStream.readInt(DataInputStream.java:387)
        at org.apache.hadoop.hbase.ipc.RpcClient$Connection.readResponse(RpcClient.java:1076)
        at org.apache.hadoop.hbase.ipc.RpcClient$Connection.run(RpcClient.java:727)

Thanks

> On 03 Sep 2015, at 15:50, Ted Yu <yu...@gmail.com> wrote:
> 
> Can you log into the master node and check master log ?
> 
> Pastebin snippet of master log if you need more help. 
> 
> Cheers
> 
> 
>> On Sep 3, 2015, at 6:24 AM, Akmal Abbasov <ak...@icloud.com> wrote:
>> 
>> Hi,
>> I’m having problems with accessing HBase master webui. when I try to access it, it returns redirect page /master-status, and then it is freezes.
>> I’ve tried to wait for a long time, but still it didn’t work.
>> I am using hbase hbase-0.98.7-hadoop2, hadoop hadoop-2.5.1
>> Any suggestions?
>> Thanks.


Re: HBase master WebUI is not responding

Posted by Akmal Abbasov <ak...@icloud.com>.
Hi Ted,
I’m getting a lot of 
2015-09-03 14:08:20,651 WARN  [1865504699@qtp-838397579-13] client.HConnectionManager$HConnectionImplementation: Checking master connection
com.google.protobuf.ServiceException: java.net.SocketTimeoutException: Call to test-m/10.10.8.53:60000 failed because java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/10.10.8.53:41243 remote=test-m/10.10.8.53:60000]
        at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1678)
        at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1719)
        at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceState.isMasterRunning(HConnectionManager.java:1508)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isKeepAliveMasterConnectedAndRunning(HConnectionManager.java:2146)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1859)
        at org.apache.hadoop.hbase.client.HBaseAdmin$MasterCallable.prepare(HBaseAdmin.java:3320)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:113)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90)
        at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3347)
        at org.apache.hadoop.hbase.client.HBaseAdmin.listTableDescriptorsByNamespace(HBaseAdmin.java:2281)
        at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.__jamon_innerUnit__catalogTables(MasterStatusTmplImpl.java:465)
        at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.renderNoFlush(MasterStatusTmplImpl.java:274)
        at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.renderNoFlush(MasterStatusTmpl.java:386)
        at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.render(MasterStatusTmpl.java:376)
        at org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:95)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
        at org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
        at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1122)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
        at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
        at java.io.FilterInputStream.read(FilterInputStream.java:133)
        at org.apache.hadoop.hbase.ipc.RpcClient$Connection$PingInputStream.read(RpcClient.java:558)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
        at java.io.DataInputStream.readInt(DataInputStream.java:387)
        at org.apache.hadoop.hbase.ipc.RpcClient$Connection.readResponse(RpcClient.java:1076)
        at org.apache.hadoop.hbase.ipc.RpcClient$Connection.run(RpcClient.java:727)

Thanks

> On 03 Sep 2015, at 15:50, Ted Yu <yu...@gmail.com> wrote:
> 
> Can you log into the master node and check master log ?
> 
> Pastebin snippet of master log if you need more help. 
> 
> Cheers
> 
> 
>> On Sep 3, 2015, at 6:24 AM, Akmal Abbasov <ak...@icloud.com> wrote:
>> 
>> Hi,
>> I’m having problems with accessing HBase master webui. when I try to access it, it returns redirect page /master-status, and then it is freezes.
>> I’ve tried to wait for a long time, but still it didn’t work.
>> I am using hbase hbase-0.98.7-hadoop2, hadoop hadoop-2.5.1
>> Any suggestions?
>> Thanks.


Re: HBase master WebUI is not responding

Posted by Akmal Abbasov <ak...@icloud.com>.
Hi Ted,
I’m getting a lot of 
2015-09-03 14:08:20,651 WARN  [1865504699@qtp-838397579-13] client.HConnectionManager$HConnectionImplementation: Checking master connection
com.google.protobuf.ServiceException: java.net.SocketTimeoutException: Call to test-m/10.10.8.53:60000 failed because java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/10.10.8.53:41243 remote=test-m/10.10.8.53:60000]
        at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1678)
        at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1719)
        at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceState.isMasterRunning(HConnectionManager.java:1508)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isKeepAliveMasterConnectedAndRunning(HConnectionManager.java:2146)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1859)
        at org.apache.hadoop.hbase.client.HBaseAdmin$MasterCallable.prepare(HBaseAdmin.java:3320)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:113)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90)
        at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3347)
        at org.apache.hadoop.hbase.client.HBaseAdmin.listTableDescriptorsByNamespace(HBaseAdmin.java:2281)
        at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.__jamon_innerUnit__catalogTables(MasterStatusTmplImpl.java:465)
        at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.renderNoFlush(MasterStatusTmplImpl.java:274)
        at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.renderNoFlush(MasterStatusTmpl.java:386)
        at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.render(MasterStatusTmpl.java:376)
        at org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:95)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
        at org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
        at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1122)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
        at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
        at java.io.FilterInputStream.read(FilterInputStream.java:133)
        at org.apache.hadoop.hbase.ipc.RpcClient$Connection$PingInputStream.read(RpcClient.java:558)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
        at java.io.DataInputStream.readInt(DataInputStream.java:387)
        at org.apache.hadoop.hbase.ipc.RpcClient$Connection.readResponse(RpcClient.java:1076)
        at org.apache.hadoop.hbase.ipc.RpcClient$Connection.run(RpcClient.java:727)

Thanks

> On 03 Sep 2015, at 15:50, Ted Yu <yu...@gmail.com> wrote:
> 
> Can you log into the master node and check master log ?
> 
> Pastebin snippet of master log if you need more help. 
> 
> Cheers
> 
> 
>> On Sep 3, 2015, at 6:24 AM, Akmal Abbasov <ak...@icloud.com> wrote:
>> 
>> Hi,
>> I’m having problems with accessing HBase master webui. when I try to access it, it returns redirect page /master-status, and then it is freezes.
>> I’ve tried to wait for a long time, but still it didn’t work.
>> I am using hbase hbase-0.98.7-hadoop2, hadoop hadoop-2.5.1
>> Any suggestions?
>> Thanks.


Re: HBase master WebUI is not responding

Posted by Akmal Abbasov <ak...@icloud.com>.
Hi Ted,
I’m getting a lot of 
2015-09-03 14:08:20,651 WARN  [1865504699@qtp-838397579-13] client.HConnectionManager$HConnectionImplementation: Checking master connection
com.google.protobuf.ServiceException: java.net.SocketTimeoutException: Call to test-m/10.10.8.53:60000 failed because java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/10.10.8.53:41243 remote=test-m/10.10.8.53:60000]
        at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1678)
        at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1719)
        at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceState.isMasterRunning(HConnectionManager.java:1508)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isKeepAliveMasterConnectedAndRunning(HConnectionManager.java:2146)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1859)
        at org.apache.hadoop.hbase.client.HBaseAdmin$MasterCallable.prepare(HBaseAdmin.java:3320)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:113)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90)
        at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3347)
        at org.apache.hadoop.hbase.client.HBaseAdmin.listTableDescriptorsByNamespace(HBaseAdmin.java:2281)
        at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.__jamon_innerUnit__catalogTables(MasterStatusTmplImpl.java:465)
        at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.renderNoFlush(MasterStatusTmplImpl.java:274)
        at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.renderNoFlush(MasterStatusTmpl.java:386)
        at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.render(MasterStatusTmpl.java:376)
        at org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:95)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
        at org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
        at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1122)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
        at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
        at java.io.FilterInputStream.read(FilterInputStream.java:133)
        at org.apache.hadoop.hbase.ipc.RpcClient$Connection$PingInputStream.read(RpcClient.java:558)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
        at java.io.DataInputStream.readInt(DataInputStream.java:387)
        at org.apache.hadoop.hbase.ipc.RpcClient$Connection.readResponse(RpcClient.java:1076)
        at org.apache.hadoop.hbase.ipc.RpcClient$Connection.run(RpcClient.java:727)

Thanks

> On 03 Sep 2015, at 15:50, Ted Yu <yu...@gmail.com> wrote:
> 
> Can you log into the master node and check master log ?
> 
> Pastebin snippet of master log if you need more help. 
> 
> Cheers
> 
> 
>> On Sep 3, 2015, at 6:24 AM, Akmal Abbasov <ak...@icloud.com> wrote:
>> 
>> Hi,
>> I’m having problems with accessing HBase master webui. when I try to access it, it returns redirect page /master-status, and then it is freezes.
>> I’ve tried to wait for a long time, but still it didn’t work.
>> I am using hbase hbase-0.98.7-hadoop2, hadoop hadoop-2.5.1
>> Any suggestions?
>> Thanks.


Re: HBase master WebUI is not responding

Posted by Ted Yu <yu...@gmail.com>.
Can you log into the master node and check master log ?

Pastebin snippet of master log if you need more help. 

Cheers


> On Sep 3, 2015, at 6:24 AM, Akmal Abbasov <ak...@icloud.com> wrote:
> 
> Hi,
> I’m having problems with accessing HBase master webui. when I try to access it, it returns redirect page /master-status, and then it is freezes.
> I’ve tried to wait for a long time, but still it didn’t work.
> I am using hbase hbase-0.98.7-hadoop2, hadoop hadoop-2.5.1
> Any suggestions?
> Thanks.

Re: HBase master WebUI is not responding

Posted by Ted Yu <yu...@gmail.com>.
Can you log into the master node and check master log ?

Pastebin snippet of master log if you need more help. 

Cheers


> On Sep 3, 2015, at 6:24 AM, Akmal Abbasov <ak...@icloud.com> wrote:
> 
> Hi,
> I’m having problems with accessing HBase master webui. when I try to access it, it returns redirect page /master-status, and then it is freezes.
> I’ve tried to wait for a long time, but still it didn’t work.
> I am using hbase hbase-0.98.7-hadoop2, hadoop hadoop-2.5.1
> Any suggestions?
> Thanks.

Re: HBase master WebUI is not responding

Posted by Ted Yu <yu...@gmail.com>.
Can you log into the master node and check master log ?

Pastebin snippet of master log if you need more help. 

Cheers


> On Sep 3, 2015, at 6:24 AM, Akmal Abbasov <ak...@icloud.com> wrote:
> 
> Hi,
> I’m having problems with accessing HBase master webui. when I try to access it, it returns redirect page /master-status, and then it is freezes.
> I’ve tried to wait for a long time, but still it didn’t work.
> I am using hbase hbase-0.98.7-hadoop2, hadoop hadoop-2.5.1
> Any suggestions?
> Thanks.

Re: HBase master WebUI is not responding

Posted by Ted Yu <yu...@gmail.com>.
Can you log into the master node and check master log ?

Pastebin snippet of master log if you need more help. 

Cheers


> On Sep 3, 2015, at 6:24 AM, Akmal Abbasov <ak...@icloud.com> wrote:
> 
> Hi,
> I’m having problems with accessing HBase master webui. when I try to access it, it returns redirect page /master-status, and then it is freezes.
> I’ve tried to wait for a long time, but still it didn’t work.
> I am using hbase hbase-0.98.7-hadoop2, hadoop hadoop-2.5.1
> Any suggestions?
> Thanks.