You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by "Tapper, Gunnar" <gu...@hp.com> on 2014/03/07 17:10:03 UTC
API to view active, completed, and failed HBase queries
Hi,
Is there an API that allows me to view active, completed, queued, and failed HBase queries beyond setting slow-query log to a low number and parsing the log files?
Thanks,
Gunnar
The person that says it cannot be done should not interrupt the person doing it.
RE: API to view active, completed, and failed HBase queries
Posted by "Tapper, Gunnar" <gu...@hp.com>.
Hi,
I can talk a bit freer about this now given that we just made HP Data Service Manager (HPDSM) for free download on http://www.vertica.com/marketplace. HPDSM targets big-data administrators (BDAs) that focus on the overall performance of workloads, space, etc. in cross-function environments.
For this specific question, I'm trying to figure out how we can help identify, visualize, and manage "rouge" HBase queries similar to the issue where a row scan took more than 10 minutes. Partly, my question is related to Trafodion (http://www.hp.com/go/trafodion; provides OLTP on top of HBase using ANSI SQL) but I am also looking at how we can provide more manageability capabilities for HBase deployments in general. In the 1.3 version, we capture HBase metrics information for historical analysis but I want to be able to help BDAs analyze the different HBase queries that the system has service and is servicing at the moment, especially when problems occur but also for capacity planning, SLA reporting, etc.
In essence, I am looking for APIs that allow me to provide the same type of workload management as HPDSM can provide for Vertica, MapReduce, and other data services.
Do such APIs exist beyond changing the limits for the slow query log and parsing the log files?
Sincerely,
Gunnar
“People don’t know what they want until you show it to them… Our task is to read things that are not yet on the page.” — Steve Jobs
-----Original Message-----
From: Jean-Marc Spaggiari [mailto:jean-marc@spaggiari.org]
Sent: Friday, March 07, 2014 10:57 AM
To: user
Subject: Re: API to view active, completed, and failed HBase queries
If it's external applications, you will want to monitor the external applications, not HBase. HBase will reply with few milliseconds, or will timeout if there is any network/infra issue. If it timeout, it's the RPC queue which will grow. I don't think it's really possible to see a request "in progress" easily in HBase. Except if you capture thread dumps very quickly.
If you have any external application running long jobs against HBase, they you might want to monitor your external application to see what it's doing and how to kill the related long running jobs if required.
JM
2014-03-07 12:30 GMT-05:00 Tapper, Gunnar <gu...@hp.com>:
> Yes and, also, situations where, for example, infrastructure problems
> and other factors impact query performance.
>
> -----Original Message-----
> From: Jean-Marc Spaggiari [mailto:jean-marc@spaggiari.org]
> Sent: Friday, March 07, 2014 10:23 AM
> To: user
> Subject: Re: API to view active, completed, and failed HBase queries
>
> Hi Gunnar,
>
> A get or a put in HBase is supposed to be VERY fast. Like, few
> milliseconds. So I'm not sure to get "being able to kill queries that
> are running too long". What is running so long? Are you talk about
> other external requests run by other tools against HBase?
>
> JM
>
>
> 2014-03-07 11:26 GMT-05:00 Tapper, Gunnar <gu...@hp.com>:
>
> > Hi Jean-Marc,
> >
> > Yes, I already have the RPC information but I want to actually view
> > and control queries including being able to kill queries that are
> > running too long or taking too many resources.
> >
> > A fail is usually a query that started (passed syntax check and
> > authorization checks) but couldn't complete for whatever reason. A
> > queued query is a query that has to wait for a resource before it's
> allowed to run.
> >
> > Thanks,
> >
> > Gunnar
> >
> > The person that says it cannot be done should not interrupt the
> > person doing it.
> >
> > -----Original Message-----
> > From: Jean-Marc Spaggiari [mailto:jean-marc@spaggiari.org]
> > Sent: Friday, March 07, 2014 9:17 AM
> > To: user
> > Subject: Re: API to view active, completed, and failed HBase queries
> >
> > Hi Gunnar,
> >
> > You can look at the metrics to see the RPC queues, that will give
> > you an idea of the active and completed requests. But I don't think
> > there is anything about "failed".
> >
> > Also, failed might have multiple sense. A get with no results, is it
> > a fail? A get against an unknown table? Or it's only when the RPC
> > did not reach the server? etc.
> >
> > JM
> >
> >
> > 2014-03-07 11:10 GMT-05:00 Tapper, Gunnar <gu...@hp.com>:
> >
> > > Hi,
> > >
> > > Is there an API that allows me to view active, completed, queued,
> > > and failed HBase queries beyond setting slow-query log to a low
> > > number and parsing the log files?
> > >
> > > Thanks,
> > >
> > > Gunnar
> > >
> > > The person that says it cannot be done should not interrupt the
> > > person doing it.
> > >
> > >
> >
>
Re: API to view active, completed, and failed HBase queries
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
If it's external applications, you will want to monitor the external
applications, not HBase. HBase will reply with few milliseconds, or will
timeout if there is any network/infra issue. If it timeout, it's the RPC
queue which will grow. I don't think it's really possible to see a request
"in progress" easily in HBase. Except if you capture thread dumps very
quickly.
If you have any external application running long jobs against HBase, they
you might want to monitor your external application to see what it's doing
and how to kill the related long running jobs if required.
JM
2014-03-07 12:30 GMT-05:00 Tapper, Gunnar <gu...@hp.com>:
> Yes and, also, situations where, for example, infrastructure problems and
> other factors impact query performance.
>
> -----Original Message-----
> From: Jean-Marc Spaggiari [mailto:jean-marc@spaggiari.org]
> Sent: Friday, March 07, 2014 10:23 AM
> To: user
> Subject: Re: API to view active, completed, and failed HBase queries
>
> Hi Gunnar,
>
> A get or a put in HBase is supposed to be VERY fast. Like, few
> milliseconds. So I'm not sure to get "being able to kill queries that are
> running too long". What is running so long? Are you talk about other
> external requests run by other tools against HBase?
>
> JM
>
>
> 2014-03-07 11:26 GMT-05:00 Tapper, Gunnar <gu...@hp.com>:
>
> > Hi Jean-Marc,
> >
> > Yes, I already have the RPC information but I want to actually view
> > and control queries including being able to kill queries that are
> > running too long or taking too many resources.
> >
> > A fail is usually a query that started (passed syntax check and
> > authorization checks) but couldn't complete for whatever reason. A
> > queued query is a query that has to wait for a resource before it's
> allowed to run.
> >
> > Thanks,
> >
> > Gunnar
> >
> > The person that says it cannot be done should not interrupt the person
> > doing it.
> >
> > -----Original Message-----
> > From: Jean-Marc Spaggiari [mailto:jean-marc@spaggiari.org]
> > Sent: Friday, March 07, 2014 9:17 AM
> > To: user
> > Subject: Re: API to view active, completed, and failed HBase queries
> >
> > Hi Gunnar,
> >
> > You can look at the metrics to see the RPC queues, that will give you
> > an idea of the active and completed requests. But I don't think there
> > is anything about "failed".
> >
> > Also, failed might have multiple sense. A get with no results, is it a
> > fail? A get against an unknown table? Or it's only when the RPC did
> > not reach the server? etc.
> >
> > JM
> >
> >
> > 2014-03-07 11:10 GMT-05:00 Tapper, Gunnar <gu...@hp.com>:
> >
> > > Hi,
> > >
> > > Is there an API that allows me to view active, completed, queued,
> > > and failed HBase queries beyond setting slow-query log to a low
> > > number and parsing the log files?
> > >
> > > Thanks,
> > >
> > > Gunnar
> > >
> > > The person that says it cannot be done should not interrupt the
> > > person doing it.
> > >
> > >
> >
>
RE: API to view active, completed, and failed HBase queries
Posted by "Tapper, Gunnar" <gu...@hp.com>.
Yes and, also, situations where, for example, infrastructure problems and other factors impact query performance.
-----Original Message-----
From: Jean-Marc Spaggiari [mailto:jean-marc@spaggiari.org]
Sent: Friday, March 07, 2014 10:23 AM
To: user
Subject: Re: API to view active, completed, and failed HBase queries
Hi Gunnar,
A get or a put in HBase is supposed to be VERY fast. Like, few milliseconds. So I'm not sure to get "being able to kill queries that are running too long". What is running so long? Are you talk about other external requests run by other tools against HBase?
JM
2014-03-07 11:26 GMT-05:00 Tapper, Gunnar <gu...@hp.com>:
> Hi Jean-Marc,
>
> Yes, I already have the RPC information but I want to actually view
> and control queries including being able to kill queries that are
> running too long or taking too many resources.
>
> A fail is usually a query that started (passed syntax check and
> authorization checks) but couldn't complete for whatever reason. A
> queued query is a query that has to wait for a resource before it's allowed to run.
>
> Thanks,
>
> Gunnar
>
> The person that says it cannot be done should not interrupt the person
> doing it.
>
> -----Original Message-----
> From: Jean-Marc Spaggiari [mailto:jean-marc@spaggiari.org]
> Sent: Friday, March 07, 2014 9:17 AM
> To: user
> Subject: Re: API to view active, completed, and failed HBase queries
>
> Hi Gunnar,
>
> You can look at the metrics to see the RPC queues, that will give you
> an idea of the active and completed requests. But I don't think there
> is anything about "failed".
>
> Also, failed might have multiple sense. A get with no results, is it a
> fail? A get against an unknown table? Or it's only when the RPC did
> not reach the server? etc.
>
> JM
>
>
> 2014-03-07 11:10 GMT-05:00 Tapper, Gunnar <gu...@hp.com>:
>
> > Hi,
> >
> > Is there an API that allows me to view active, completed, queued,
> > and failed HBase queries beyond setting slow-query log to a low
> > number and parsing the log files?
> >
> > Thanks,
> >
> > Gunnar
> >
> > The person that says it cannot be done should not interrupt the
> > person doing it.
> >
> >
>
Re: API to view active, completed, and failed HBase queries
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Hi Gunnar,
A get or a put in HBase is supposed to be VERY fast. Like, few
milliseconds. So I'm not sure to get "being able to kill queries that are
running too long". What is running so long? Are you talk about other
external requests run by other tools against HBase?
JM
2014-03-07 11:26 GMT-05:00 Tapper, Gunnar <gu...@hp.com>:
> Hi Jean-Marc,
>
> Yes, I already have the RPC information but I want to actually view and
> control queries including being able to kill queries that are running too
> long or taking too many resources.
>
> A fail is usually a query that started (passed syntax check and
> authorization checks) but couldn't complete for whatever reason. A queued
> query is a query that has to wait for a resource before it's allowed to run.
>
> Thanks,
>
> Gunnar
>
> The person that says it cannot be done should not interrupt the person
> doing it.
>
> -----Original Message-----
> From: Jean-Marc Spaggiari [mailto:jean-marc@spaggiari.org]
> Sent: Friday, March 07, 2014 9:17 AM
> To: user
> Subject: Re: API to view active, completed, and failed HBase queries
>
> Hi Gunnar,
>
> You can look at the metrics to see the RPC queues, that will give you an
> idea of the active and completed requests. But I don't think there is
> anything about "failed".
>
> Also, failed might have multiple sense. A get with no results, is it a
> fail? A get against an unknown table? Or it's only when the RPC did not
> reach the server? etc.
>
> JM
>
>
> 2014-03-07 11:10 GMT-05:00 Tapper, Gunnar <gu...@hp.com>:
>
> > Hi,
> >
> > Is there an API that allows me to view active, completed, queued, and
> > failed HBase queries beyond setting slow-query log to a low number and
> > parsing the log files?
> >
> > Thanks,
> >
> > Gunnar
> >
> > The person that says it cannot be done should not interrupt the person
> > doing it.
> >
> >
>
Re: API to view active, completed, and failed HBase queries
Posted by Vijayakumar Ramdoss <ne...@live.com>.
A newbie question , how can I look the RPC queue details
Sent from my iPhone
> On Mar 7, 2014, at 11:27 AM, "Tapper, Gunnar" <gu...@hp.com> wrote:
>
> Hi Jean-Marc,
>
> Yes, I already have the RPC information but I want to actually view and control queries including being able to kill queries that are running too long or taking too many resources.
>
> A fail is usually a query that started (passed syntax check and authorization checks) but couldn't complete for whatever reason. A queued query is a query that has to wait for a resource before it's allowed to run.
>
> Thanks,
>
> Gunnar
>
> The person that says it cannot be done should not interrupt the person doing it.
>
> -----Original Message-----
> From: Jean-Marc Spaggiari [mailto:jean-marc@spaggiari.org]
> Sent: Friday, March 07, 2014 9:17 AM
> To: user
> Subject: Re: API to view active, completed, and failed HBase queries
>
> Hi Gunnar,
>
> You can look at the metrics to see the RPC queues, that will give you an idea of the active and completed requests. But I don't think there is anything about "failed".
>
> Also, failed might have multiple sense. A get with no results, is it a fail? A get against an unknown table? Or it's only when the RPC did not reach the server? etc.
>
> JM
>
>
> 2014-03-07 11:10 GMT-05:00 Tapper, Gunnar <gu...@hp.com>:
>
>> Hi,
>>
>> Is there an API that allows me to view active, completed, queued, and
>> failed HBase queries beyond setting slow-query log to a low number and
>> parsing the log files?
>>
>> Thanks,
>>
>> Gunnar
>>
>> The person that says it cannot be done should not interrupt the person
>> doing it.
>>
>>
RE: API to view active, completed, and failed HBase queries
Posted by "Tapper, Gunnar" <gu...@hp.com>.
Hi Jean-Marc,
Yes, I already have the RPC information but I want to actually view and control queries including being able to kill queries that are running too long or taking too many resources.
A fail is usually a query that started (passed syntax check and authorization checks) but couldn't complete for whatever reason. A queued query is a query that has to wait for a resource before it's allowed to run.
Thanks,
Gunnar
The person that says it cannot be done should not interrupt the person doing it.
-----Original Message-----
From: Jean-Marc Spaggiari [mailto:jean-marc@spaggiari.org]
Sent: Friday, March 07, 2014 9:17 AM
To: user
Subject: Re: API to view active, completed, and failed HBase queries
Hi Gunnar,
You can look at the metrics to see the RPC queues, that will give you an idea of the active and completed requests. But I don't think there is anything about "failed".
Also, failed might have multiple sense. A get with no results, is it a fail? A get against an unknown table? Or it's only when the RPC did not reach the server? etc.
JM
2014-03-07 11:10 GMT-05:00 Tapper, Gunnar <gu...@hp.com>:
> Hi,
>
> Is there an API that allows me to view active, completed, queued, and
> failed HBase queries beyond setting slow-query log to a low number and
> parsing the log files?
>
> Thanks,
>
> Gunnar
>
> The person that says it cannot be done should not interrupt the person
> doing it.
>
>
Re: API to view active, completed, and failed HBase queries
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Hi Gunnar,
You can look at the metrics to see the RPC queues, that will give you an
idea of the active and completed requests. But I don't think there is
anything about "failed".
Also, failed might have multiple sense. A get with no results, is it a
fail? A get against an unknown table? Or it's only when the RPC did not
reach the server? etc.
JM
2014-03-07 11:10 GMT-05:00 Tapper, Gunnar <gu...@hp.com>:
> Hi,
>
> Is there an API that allows me to view active, completed, queued, and
> failed HBase queries beyond setting slow-query log to a low number and
> parsing the log files?
>
> Thanks,
>
> Gunnar
>
> The person that says it cannot be done should not interrupt the person
> doing it.
>
>