You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Stack <st...@duboce.net> on 2014/09/08 20:55:49 UTC
Doing some testing of branch-1 and want to know what replica messages mean?
I'm doing some basic testing of branch-1 to see if can catch any ugly stuff
before we cut our first 0.99.
Can someone help me w/ following debug-level messages:
2014-09-08 11:24:13,739 DEBUG [TestClient-43]
client.ScannerCallableWithReplicas: Setting current scanner as -1
associated with 0
2014-09-08 11:24:13,739 DEBUG [TestClient-76]
client.ScannerCallableWithReplicas: Setting current scanner as -1
associated with 0
2014-09-08 11:24:13,739 DEBUG [TestClient-93]
client.ScannerCallableWithReplicas: Setting current scanner as -1
associated with 0
2014-09-08 11:24:13,748 DEBUG [TestClient-10]
client.ScannerCallableWithReplicas: Setting current scanner as -1
associated with 0
The -1 is the scanner id it seems but I'd think a -1 id is 'bad'... least
that is what I think when I see it. I'm at DEBUG level so not so bad but
there is a lot of replica logging going on. Should this logging be
switched? i.e. only log when we are doing replicas? Is there a config. to
check?
Thanks,
St.Ack
Re: Doing some testing of branch-1 and want to know what replica
messages mean?
Posted by Enis Söztutar <en...@gmail.com>.
Hey Stack,
I think I have a patch somewhere that reduces that logging to TRACE level.
It is not related to replicas, but for the patch
https://issues.apache.org/jira/browse/HBASE-10357, we've made the Scan code
paths for replicas and regular scans the same (that is why it is named
ScannerCallableWithReplica) so that the code is a bit better organized.
Let me put that patch in jira.
Enis
On Mon, Sep 8, 2014 at 11:55 AM, Stack <st...@duboce.net> wrote:
> I'm doing some basic testing of branch-1 to see if can catch any ugly stuff
> before we cut our first 0.99.
>
> Can someone help me w/ following debug-level messages:
>
> 2014-09-08 11:24:13,739 DEBUG [TestClient-43]
> client.ScannerCallableWithReplicas: Setting current scanner as -1
> associated with 0
> 2014-09-08 11:24:13,739 DEBUG [TestClient-76]
> client.ScannerCallableWithReplicas: Setting current scanner as -1
> associated with 0
> 2014-09-08 11:24:13,739 DEBUG [TestClient-93]
> client.ScannerCallableWithReplicas: Setting current scanner as -1
> associated with 0
> 2014-09-08 11:24:13,748 DEBUG [TestClient-10]
> client.ScannerCallableWithReplicas: Setting current scanner as -1
> associated with 0
>
> The -1 is the scanner id it seems but I'd think a -1 id is 'bad'... least
> that is what I think when I see it. I'm at DEBUG level so not so bad but
> there is a lot of replica logging going on. Should this logging be
> switched? i.e. only log when we are doing replicas? Is there a config. to
> check?
>
> Thanks,
> St.Ack
>
Re: Doing some testing of branch-1 and want to know what replica
messages mean?
Posted by Stack <st...@duboce.net>.
Thanks E. Agree trace would be better (-1 seems to be closed scanner or
uninitialized... and we are running through this loop to do cleanup of
outstanding scanners so the message is a little odd.... better at
trace-level). Let me look at what you've posted...
St.Ack
On Mon, Sep 8, 2014 at 2:24 PM, Enis Söztutar <en...@gmail.com> wrote:
> This maybe from recent changes from Nicolas in the client side. Yeah, we
> should not be logging at the INFO level.
>
> Enis
>
>
> On Mon, Sep 8, 2014 at 12:14 PM, Stack <st...@duboce.net> wrote:
>
> > There is a load of this stuff too at INFO level on client-side:
> >
> > 2014-09-08 11:24:27,533 INFO [htable-pool25-t1] client.AsyncProcess:
> #129,
> > table=TestTable, attempt=2/35 failed 224 ops, last exception:
> > org.apache.hadoop.hbase.NotServingRegionException:
> > org.apache.hadoop.hbase.NotServingRegionException: Region
> >
> >
> TestTable,00000000000000000000283916,1410199679935.3fc14391d0ecfdb59950c62e76367cc4.
> > is not online on c2021.halxg.cloudera.com,16020,1410200244722
> > at
> >
> >
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2605)
> > at
> >
> >
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:796)
> > at
> >
> >
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1711)
> > at
> >
> >
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:30090)
> > at
> org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2014)
> > at
> org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
> > at
> >
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114)
> > at
> > org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94)
> > at java.lang.Thread.run(Thread.java:744)
> >
> > Seeing exceptions in your log will disorientate (The NSRE looks like it
> has
> > no associated 'message' on the client-side).
> >
> > Anyone looking into fixing these?
> >
> > Thanks,
> > St.Ack
> >
> >
> > On Mon, Sep 8, 2014 at 11:55 AM, Stack <st...@duboce.net> wrote:
> >
> > > I'm doing some basic testing of branch-1 to see if can catch any ugly
> > > stuff before we cut our first 0.99.
> > >
> > > Can someone help me w/ following debug-level messages:
> > >
> > > 2014-09-08 11:24:13,739 DEBUG [TestClient-43]
> > > client.ScannerCallableWithReplicas: Setting current scanner as -1
> > > associated with 0
> > > 2014-09-08 11:24:13,739 DEBUG [TestClient-76]
> > > client.ScannerCallableWithReplicas: Setting current scanner as -1
> > > associated with 0
> > > 2014-09-08 11:24:13,739 DEBUG [TestClient-93]
> > > client.ScannerCallableWithReplicas: Setting current scanner as -1
> > > associated with 0
> > > 2014-09-08 11:24:13,748 DEBUG [TestClient-10]
> > > client.ScannerCallableWithReplicas: Setting current scanner as -1
> > > associated with 0
> > >
> > > The -1 is the scanner id it seems but I'd think a -1 id is 'bad'...
> least
> > > that is what I think when I see it. I'm at DEBUG level so not so bad
> but
> > > there is a lot of replica logging going on. Should this logging be
> > > switched? i.e. only log when we are doing replicas? Is there a config.
> > to
> > > check?
> > >
> > > Thanks,
> > > St.Ack
> > >
> > >
> > >
> >
>
Re: Doing some testing of branch-1 and want to know what replica
messages mean?
Posted by Enis Söztutar <en...@gmail.com>.
This maybe from recent changes from Nicolas in the client side. Yeah, we
should not be logging at the INFO level.
Enis
On Mon, Sep 8, 2014 at 12:14 PM, Stack <st...@duboce.net> wrote:
> There is a load of this stuff too at INFO level on client-side:
>
> 2014-09-08 11:24:27,533 INFO [htable-pool25-t1] client.AsyncProcess: #129,
> table=TestTable, attempt=2/35 failed 224 ops, last exception:
> org.apache.hadoop.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException: Region
>
> TestTable,00000000000000000000283916,1410199679935.3fc14391d0ecfdb59950c62e76367cc4.
> is not online on c2021.halxg.cloudera.com,16020,1410200244722
> at
>
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2605)
> at
>
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:796)
> at
>
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1711)
> at
>
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:30090)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2014)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94)
> at java.lang.Thread.run(Thread.java:744)
>
> Seeing exceptions in your log will disorientate (The NSRE looks like it has
> no associated 'message' on the client-side).
>
> Anyone looking into fixing these?
>
> Thanks,
> St.Ack
>
>
> On Mon, Sep 8, 2014 at 11:55 AM, Stack <st...@duboce.net> wrote:
>
> > I'm doing some basic testing of branch-1 to see if can catch any ugly
> > stuff before we cut our first 0.99.
> >
> > Can someone help me w/ following debug-level messages:
> >
> > 2014-09-08 11:24:13,739 DEBUG [TestClient-43]
> > client.ScannerCallableWithReplicas: Setting current scanner as -1
> > associated with 0
> > 2014-09-08 11:24:13,739 DEBUG [TestClient-76]
> > client.ScannerCallableWithReplicas: Setting current scanner as -1
> > associated with 0
> > 2014-09-08 11:24:13,739 DEBUG [TestClient-93]
> > client.ScannerCallableWithReplicas: Setting current scanner as -1
> > associated with 0
> > 2014-09-08 11:24:13,748 DEBUG [TestClient-10]
> > client.ScannerCallableWithReplicas: Setting current scanner as -1
> > associated with 0
> >
> > The -1 is the scanner id it seems but I'd think a -1 id is 'bad'... least
> > that is what I think when I see it. I'm at DEBUG level so not so bad but
> > there is a lot of replica logging going on. Should this logging be
> > switched? i.e. only log when we are doing replicas? Is there a config.
> to
> > check?
> >
> > Thanks,
> > St.Ack
> >
> >
> >
>
Re: Doing some testing of branch-1 and want to know what replica
messages mean?
Posted by Stack <st...@duboce.net>.
There is a load of this stuff too at INFO level on client-side:
2014-09-08 11:24:27,533 INFO [htable-pool25-t1] client.AsyncProcess: #129,
table=TestTable, attempt=2/35 failed 224 ops, last exception:
org.apache.hadoop.hbase.NotServingRegionException:
org.apache.hadoop.hbase.NotServingRegionException: Region
TestTable,00000000000000000000283916,1410199679935.3fc14391d0ecfdb59950c62e76367cc4.
is not online on c2021.halxg.cloudera.com,16020,1410200244722
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2605)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:796)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1711)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:30090)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2014)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94)
at java.lang.Thread.run(Thread.java:744)
Seeing exceptions in your log will disorientate (The NSRE looks like it has
no associated 'message' on the client-side).
Anyone looking into fixing these?
Thanks,
St.Ack
On Mon, Sep 8, 2014 at 11:55 AM, Stack <st...@duboce.net> wrote:
> I'm doing some basic testing of branch-1 to see if can catch any ugly
> stuff before we cut our first 0.99.
>
> Can someone help me w/ following debug-level messages:
>
> 2014-09-08 11:24:13,739 DEBUG [TestClient-43]
> client.ScannerCallableWithReplicas: Setting current scanner as -1
> associated with 0
> 2014-09-08 11:24:13,739 DEBUG [TestClient-76]
> client.ScannerCallableWithReplicas: Setting current scanner as -1
> associated with 0
> 2014-09-08 11:24:13,739 DEBUG [TestClient-93]
> client.ScannerCallableWithReplicas: Setting current scanner as -1
> associated with 0
> 2014-09-08 11:24:13,748 DEBUG [TestClient-10]
> client.ScannerCallableWithReplicas: Setting current scanner as -1
> associated with 0
>
> The -1 is the scanner id it seems but I'd think a -1 id is 'bad'... least
> that is what I think when I see it. I'm at DEBUG level so not so bad but
> there is a lot of replica logging going on. Should this logging be
> switched? i.e. only log when we are doing replicas? Is there a config. to
> check?
>
> Thanks,
> St.Ack
>
>
>