You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Mike Spreitzer <ms...@us.ibm.com> on 2012/04/26 19:55:52 UTC
Understanding responseTooSlow in HBase regionserver log
I found http://hbase.apache.org/book/ops.monitoring.html but am confused
about the distinction between operationTooSlow and responseTooSlow. I
find the text "client operation" ambiguous, I am not sure whether that
means the side emitting this log entry is the client and the slow
operation happened on some server elsewhere or it is the other way around
or something else. I am running HBase-0.92.0 and found the following in
the log of a regionserver:
2012-04-26 13:17:30,698 WARN org.apache.hadoop.ipc.HBaseServer:
(responseTooSlow):
{"processingtimems":10340,"call":"execCoprocessor([B@f2d376,
<mycoprocessor/>, rpc version=1, client version=0, methodsFingerPrint=0),
rpc version=1, client version=29,
methodsFingerPrint=54742778","client":"10.12.203.173:58531","starttimems":1335460640355,"queuetimems":0,"class":"HRegionServer","responsesize":0,"method":"execCoprocessor"}
10.12.203.173 is not the IP address of the machine running this
regionserver, it is the IP address of the machine running my HBase client
application.
When is operationTooSlow used and when is responseTooSlow used?
In my example above, am I correct that this was merely a warning issued on
the regionserver saying that my coprocessor took a bit over 10 seconds on
that regionserver and this did not cause any actual harm? Note that my
HBase client application printed no complaints and appeared to work
correctly (although my evaluation of "correctness" was limited).
Thanks,
Mike
Re: Understanding responseTooSlow in HBase regionserver log
Posted by Shrijeet Paliwal <sh...@rocketfuel.com>.
Mike,
Check logResponse method
of org.apache.hadoop.hbase.ipc.WritableRpcEngine.Server.
This piece in particular :
else if (params.length == 1 && instance instanceof HRegionServer &&
params[0] instanceof Operation) {
// annotate the response map with operation details
responseInfo.putAll(((Operation) params[0]).toMap());
// report to the log file
LOG.warn("(*operation*" + tag + "): " +
mapper.writeValueAsString(responseInfo));
} else {
// can't get JSON details, so just report call.toString() along with
// a more generic tag.
responseInfo.put("call", call.toString());
LOG.warn("(*response*" + tag + "): " +
mapper.writeValueAsString(responseInfo));
}
On Thu, Apr 26, 2012 at 11:31 AM, Jean-Daniel Cryans <jd...@apache.org>
wrote:
>
> You are right.
>
> FWIW I cannot find operationTooSlow in the source code, not sure
> what's going on there.
>
> J-D
>
> On Thu, Apr 26, 2012 at 10:55 AM, Mike Spreitzer <ms...@us.ibm.com>
wrote:
> > I found http://hbase.apache.org/book/ops.monitoring.html but am confused
> > about the distinction between operationTooSlow and responseTooSlow. I
> > find the text "client operation" ambiguous, I am not sure whether that
> > means the side emitting this log entry is the client and the slow
> > operation happened on some server elsewhere or it is the other way
around
> > or something else. I am running HBase-0.92.0 and found the following in
> > the log of a regionserver:
> >
> > 2012-04-26 13:17:30,698 WARN org.apache.hadoop.ipc.HBaseServer:
> > (responseTooSlow):
> > {"processingtimems":10340,"call":"execCoprocessor([B@f2d376,
> > <mycoprocessor/>, rpc version=1, client version=0,
methodsFingerPrint=0),
> > rpc version=1, client version=29,
> > methodsFingerPrint=54742778","client":"10.12.203.173:58531
","starttimems":1335460640355,"queuetimems":0,"class":"HRegionServer","responsesize":0,"method":"execCoprocessor"}
> >
> > 10.12.203.173 is not the IP address of the machine running this
> > regionserver, it is the IP address of the machine running my HBase
client
> > application.
> >
> > When is operationTooSlow used and when is responseTooSlow used?
> >
> > In my example above, am I correct that this was merely a warning issued
on
> > the regionserver saying that my coprocessor took a bit over 10 seconds
on
> > that regionserver and this did not cause any actual harm? Note that my
> > HBase client application printed no complaints and appeared to work
> > correctly (although my evaluation of "correctness" was limited).
> >
> > Thanks,
> > Mike
Re: Understanding responseTooSlow in HBase regionserver log
Posted by Jean-Daniel Cryans <jd...@apache.org>.
You are right.
FWIW I cannot find operationTooSlow in the source code, not sure
what's going on there.
J-D
On Thu, Apr 26, 2012 at 10:55 AM, Mike Spreitzer <ms...@us.ibm.com> wrote:
> I found http://hbase.apache.org/book/ops.monitoring.html but am confused
> about the distinction between operationTooSlow and responseTooSlow. I
> find the text "client operation" ambiguous, I am not sure whether that
> means the side emitting this log entry is the client and the slow
> operation happened on some server elsewhere or it is the other way around
> or something else. I am running HBase-0.92.0 and found the following in
> the log of a regionserver:
>
> 2012-04-26 13:17:30,698 WARN org.apache.hadoop.ipc.HBaseServer:
> (responseTooSlow):
> {"processingtimems":10340,"call":"execCoprocessor([B@f2d376,
> <mycoprocessor/>, rpc version=1, client version=0, methodsFingerPrint=0),
> rpc version=1, client version=29,
> methodsFingerPrint=54742778","client":"10.12.203.173:58531","starttimems":1335460640355,"queuetimems":0,"class":"HRegionServer","responsesize":0,"method":"execCoprocessor"}
>
> 10.12.203.173 is not the IP address of the machine running this
> regionserver, it is the IP address of the machine running my HBase client
> application.
>
> When is operationTooSlow used and when is responseTooSlow used?
>
> In my example above, am I correct that this was merely a warning issued on
> the regionserver saying that my coprocessor took a bit over 10 seconds on
> that regionserver and this did not cause any actual harm? Note that my
> HBase client application printed no complaints and appeared to work
> correctly (although my evaluation of "correctness" was limited).
>
> Thanks,
> Mike