You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Anup Tiwari <an...@games24x7.com> on 2016/08/18 07:34:27 UTC
Fetch queries status from drill prompt
Hi All,
We want to see all types of queries which ran on drill cluster or currently
running from drill prompt, Can someone help us on this?
To achieve above , we read the drill documentation and set up a storage
plugin to access local file system and able to query
*"drillbit_queries.json"* log file, but in above file we are getting status
of all queries whose status is either "cancelled","completed" or "failed"
but missing "running". At the same time we check drill profile interface
where we can see running queries.
I am sure if we can see on User-Interface then it must be coming from
somewhere.
Kindly help me on this.
Regards,
*Anup Tiwari*
Software Engineer(BI-Team),PlayGames24x7 Pvt Ltd
Re: Fetch queries status from drill prompt
Posted by Anup Tiwari <an...@games24x7.com>.
thanks for link.. but till then is their any other way? OR way to read
zookeper logs in drill, as we are showing on profile UI?
Regards,
*Anup Tiwari*
Software Engineer(BI-Team),PlayGames24x7 Pvt Ltd
On Fri, Aug 19, 2016 at 6:40 PM, Khurram Faraaz <kf...@maprtech.com>
wrote:
> There is an open JIRA DRILL-4258
> <https://issues.apache.org/jira/browse/DRILL-4258>
> Once that is fixed/implemented you can query from sys.queries or
> sys.statements to know about all SQL statements that are in different
> states.
>
> Khurram
>
> On Fri, Aug 19, 2016 at 4:45 PM, Anup Tiwari <an...@games24x7.com>
> wrote:
>
> > Thanks for above info.. i found running queries information in Zookeeper
> > logs but i guess it is in binary or some other form.
> > Is their any way to read it in drill?
> >
> > Regards,
> > *Anup Tiwari*
> > Software Engineer(BI-Team),PlayGames24x7 Pvt Ltd
> >
> > On Thu, Aug 18, 2016 at 11:59 PM, Sudheesh Katkam <sk...@maprtech.com>
> > wrote:
> >
> > > Profiles of running queries are stored in Zookeeper (or the configured
> > > transient store).
> > >
> > > Thank you,
> > > Sudheesh
> > >
> > > > On Aug 18, 2016, at 11:23 AM, Anup Tiwari <anup.tiwari@games24x7.com
> >
> > > wrote:
> > > >
> > > > Thanks chun for info..
> > > >
> > > > But can you tell me from where, running queries status come on
> profile
> > > > user-interface(UI)? Because if it's coming on profile UI then it must
> > > have
> > > > some back end file or something like that..
> > > > On 18-Aug-2016 11:37 PM, "Chun Chang" <cc...@maprtech.com> wrote:
> > > >
> > > > Anup,
> > > >
> > > > I believe only when a query is in a "terminal", i.e.
> > > > cancelled/completed/failed state, then it is written to the
> > > > drillbit_queries.json file on the foreman node. If what you want to
> do
> > is
> > > > monitoring queries running on your cluster, your best bet is to
> > configure
> > > > your cluster to store profile information on HDFS and monitor through
> > > query
> > > > profile. Remember if you have a cluster, you will have a
> > > > drillbit_queries.json file on very cluster node where drillbit is
> > > running.
> > > > And each file only contains completed queries that were run on that
> > node
> > > as
> > > > foreman. You would have to aggregate to get the whole picture of your
> > > > cluster. Even that, you will not see running queries.
> > > >
> > > > Hope this helps.
> > > >
> > > > On Thu, Aug 18, 2016 at 12:34 AM, Anup Tiwari <
> > anup.tiwari@games24x7.com
> > > >
> > > > wrote:
> > > >
> > > >> Hi All,
> > > >>
> > > >> We want to see all types of queries which ran on drill cluster or
> > > > currently
> > > >> running from drill prompt, Can someone help us on this?
> > > >>
> > > >> To achieve above , we read the drill documentation and set up a
> > storage
> > > >> plugin to access local file system and able to query
> > > >> *"drillbit_queries.json"* log file, but in above file we are getting
> > > > status
> > > >> of all queries whose status is either "cancelled","completed" or
> > > "failed"
> > > >> but missing "running". At the same time we check drill profile
> > interface
> > > >> where we can see running queries.
> > > >>
> > > >> I am sure if we can see on User-Interface then it must be coming
> from
> > > >> somewhere.
> > > >>
> > > >> Kindly help me on this.
> > > >>
> > > >> Regards,
> > > >> *Anup Tiwari*
> > > >> Software Engineer(BI-Team),PlayGames24x7 Pvt Ltd
> > > >>
> > >
> > >
> >
>
Re: Fetch queries status from drill prompt
Posted by Khurram Faraaz <kf...@maprtech.com>.
There is an open JIRA DRILL-4258
<https://issues.apache.org/jira/browse/DRILL-4258>
Once that is fixed/implemented you can query from sys.queries or
sys.statements to know about all SQL statements that are in different
states.
Khurram
On Fri, Aug 19, 2016 at 4:45 PM, Anup Tiwari <an...@games24x7.com>
wrote:
> Thanks for above info.. i found running queries information in Zookeeper
> logs but i guess it is in binary or some other form.
> Is their any way to read it in drill?
>
> Regards,
> *Anup Tiwari*
> Software Engineer(BI-Team),PlayGames24x7 Pvt Ltd
>
> On Thu, Aug 18, 2016 at 11:59 PM, Sudheesh Katkam <sk...@maprtech.com>
> wrote:
>
> > Profiles of running queries are stored in Zookeeper (or the configured
> > transient store).
> >
> > Thank you,
> > Sudheesh
> >
> > > On Aug 18, 2016, at 11:23 AM, Anup Tiwari <an...@games24x7.com>
> > wrote:
> > >
> > > Thanks chun for info..
> > >
> > > But can you tell me from where, running queries status come on profile
> > > user-interface(UI)? Because if it's coming on profile UI then it must
> > have
> > > some back end file or something like that..
> > > On 18-Aug-2016 11:37 PM, "Chun Chang" <cc...@maprtech.com> wrote:
> > >
> > > Anup,
> > >
> > > I believe only when a query is in a "terminal", i.e.
> > > cancelled/completed/failed state, then it is written to the
> > > drillbit_queries.json file on the foreman node. If what you want to do
> is
> > > monitoring queries running on your cluster, your best bet is to
> configure
> > > your cluster to store profile information on HDFS and monitor through
> > query
> > > profile. Remember if you have a cluster, you will have a
> > > drillbit_queries.json file on very cluster node where drillbit is
> > running.
> > > And each file only contains completed queries that were run on that
> node
> > as
> > > foreman. You would have to aggregate to get the whole picture of your
> > > cluster. Even that, you will not see running queries.
> > >
> > > Hope this helps.
> > >
> > > On Thu, Aug 18, 2016 at 12:34 AM, Anup Tiwari <
> anup.tiwari@games24x7.com
> > >
> > > wrote:
> > >
> > >> Hi All,
> > >>
> > >> We want to see all types of queries which ran on drill cluster or
> > > currently
> > >> running from drill prompt, Can someone help us on this?
> > >>
> > >> To achieve above , we read the drill documentation and set up a
> storage
> > >> plugin to access local file system and able to query
> > >> *"drillbit_queries.json"* log file, but in above file we are getting
> > > status
> > >> of all queries whose status is either "cancelled","completed" or
> > "failed"
> > >> but missing "running". At the same time we check drill profile
> interface
> > >> where we can see running queries.
> > >>
> > >> I am sure if we can see on User-Interface then it must be coming from
> > >> somewhere.
> > >>
> > >> Kindly help me on this.
> > >>
> > >> Regards,
> > >> *Anup Tiwari*
> > >> Software Engineer(BI-Team),PlayGames24x7 Pvt Ltd
> > >>
> >
> >
>
Re: Fetch queries status from drill prompt
Posted by Anup Tiwari <an...@games24x7.com>.
Thanks for above info.. i found running queries information in Zookeeper
logs but i guess it is in binary or some other form.
Is their any way to read it in drill?
Regards,
*Anup Tiwari*
Software Engineer(BI-Team),PlayGames24x7 Pvt Ltd
On Thu, Aug 18, 2016 at 11:59 PM, Sudheesh Katkam <sk...@maprtech.com>
wrote:
> Profiles of running queries are stored in Zookeeper (or the configured
> transient store).
>
> Thank you,
> Sudheesh
>
> > On Aug 18, 2016, at 11:23 AM, Anup Tiwari <an...@games24x7.com>
> wrote:
> >
> > Thanks chun for info..
> >
> > But can you tell me from where, running queries status come on profile
> > user-interface(UI)? Because if it's coming on profile UI then it must
> have
> > some back end file or something like that..
> > On 18-Aug-2016 11:37 PM, "Chun Chang" <cc...@maprtech.com> wrote:
> >
> > Anup,
> >
> > I believe only when a query is in a "terminal", i.e.
> > cancelled/completed/failed state, then it is written to the
> > drillbit_queries.json file on the foreman node. If what you want to do is
> > monitoring queries running on your cluster, your best bet is to configure
> > your cluster to store profile information on HDFS and monitor through
> query
> > profile. Remember if you have a cluster, you will have a
> > drillbit_queries.json file on very cluster node where drillbit is
> running.
> > And each file only contains completed queries that were run on that node
> as
> > foreman. You would have to aggregate to get the whole picture of your
> > cluster. Even that, you will not see running queries.
> >
> > Hope this helps.
> >
> > On Thu, Aug 18, 2016 at 12:34 AM, Anup Tiwari <anup.tiwari@games24x7.com
> >
> > wrote:
> >
> >> Hi All,
> >>
> >> We want to see all types of queries which ran on drill cluster or
> > currently
> >> running from drill prompt, Can someone help us on this?
> >>
> >> To achieve above , we read the drill documentation and set up a storage
> >> plugin to access local file system and able to query
> >> *"drillbit_queries.json"* log file, but in above file we are getting
> > status
> >> of all queries whose status is either "cancelled","completed" or
> "failed"
> >> but missing "running". At the same time we check drill profile interface
> >> where we can see running queries.
> >>
> >> I am sure if we can see on User-Interface then it must be coming from
> >> somewhere.
> >>
> >> Kindly help me on this.
> >>
> >> Regards,
> >> *Anup Tiwari*
> >> Software Engineer(BI-Team),PlayGames24x7 Pvt Ltd
> >>
>
>
Re: Fetch queries status from drill prompt
Posted by Sudheesh Katkam <sk...@maprtech.com>.
Profiles of running queries are stored in Zookeeper (or the configured transient store).
Thank you,
Sudheesh
> On Aug 18, 2016, at 11:23 AM, Anup Tiwari <an...@games24x7.com> wrote:
>
> Thanks chun for info..
>
> But can you tell me from where, running queries status come on profile
> user-interface(UI)? Because if it's coming on profile UI then it must have
> some back end file or something like that..
> On 18-Aug-2016 11:37 PM, "Chun Chang" <cc...@maprtech.com> wrote:
>
> Anup,
>
> I believe only when a query is in a "terminal", i.e.
> cancelled/completed/failed state, then it is written to the
> drillbit_queries.json file on the foreman node. If what you want to do is
> monitoring queries running on your cluster, your best bet is to configure
> your cluster to store profile information on HDFS and monitor through query
> profile. Remember if you have a cluster, you will have a
> drillbit_queries.json file on very cluster node where drillbit is running.
> And each file only contains completed queries that were run on that node as
> foreman. You would have to aggregate to get the whole picture of your
> cluster. Even that, you will not see running queries.
>
> Hope this helps.
>
> On Thu, Aug 18, 2016 at 12:34 AM, Anup Tiwari <an...@games24x7.com>
> wrote:
>
>> Hi All,
>>
>> We want to see all types of queries which ran on drill cluster or
> currently
>> running from drill prompt, Can someone help us on this?
>>
>> To achieve above , we read the drill documentation and set up a storage
>> plugin to access local file system and able to query
>> *"drillbit_queries.json"* log file, but in above file we are getting
> status
>> of all queries whose status is either "cancelled","completed" or "failed"
>> but missing "running". At the same time we check drill profile interface
>> where we can see running queries.
>>
>> I am sure if we can see on User-Interface then it must be coming from
>> somewhere.
>>
>> Kindly help me on this.
>>
>> Regards,
>> *Anup Tiwari*
>> Software Engineer(BI-Team),PlayGames24x7 Pvt Ltd
>>
Re: Fetch queries status from drill prompt
Posted by Anup Tiwari <an...@games24x7.com>.
Thanks chun for info..
But can you tell me from where, running queries status come on profile
user-interface(UI)? Because if it's coming on profile UI then it must have
some back end file or something like that..
On 18-Aug-2016 11:37 PM, "Chun Chang" <cc...@maprtech.com> wrote:
Anup,
I believe only when a query is in a "terminal", i.e.
cancelled/completed/failed state, then it is written to the
drillbit_queries.json file on the foreman node. If what you want to do is
monitoring queries running on your cluster, your best bet is to configure
your cluster to store profile information on HDFS and monitor through query
profile. Remember if you have a cluster, you will have a
drillbit_queries.json file on very cluster node where drillbit is running.
And each file only contains completed queries that were run on that node as
foreman. You would have to aggregate to get the whole picture of your
cluster. Even that, you will not see running queries.
Hope this helps.
On Thu, Aug 18, 2016 at 12:34 AM, Anup Tiwari <an...@games24x7.com>
wrote:
> Hi All,
>
> We want to see all types of queries which ran on drill cluster or
currently
> running from drill prompt, Can someone help us on this?
>
> To achieve above , we read the drill documentation and set up a storage
> plugin to access local file system and able to query
> *"drillbit_queries.json"* log file, but in above file we are getting
status
> of all queries whose status is either "cancelled","completed" or "failed"
> but missing "running". At the same time we check drill profile interface
> where we can see running queries.
>
> I am sure if we can see on User-Interface then it must be coming from
> somewhere.
>
> Kindly help me on this.
>
> Regards,
> *Anup Tiwari*
> Software Engineer(BI-Team),PlayGames24x7 Pvt Ltd
>
Re: Fetch queries status from drill prompt
Posted by Chun Chang <cc...@maprtech.com>.
Anup,
I believe only when a query is in a "terminal", i.e.
cancelled/completed/failed state, then it is written to the
drillbit_queries.json file on the foreman node. If what you want to do is
monitoring queries running on your cluster, your best bet is to configure
your cluster to store profile information on HDFS and monitor through query
profile. Remember if you have a cluster, you will have a
drillbit_queries.json file on very cluster node where drillbit is running.
And each file only contains completed queries that were run on that node as
foreman. You would have to aggregate to get the whole picture of your
cluster. Even that, you will not see running queries.
Hope this helps.
On Thu, Aug 18, 2016 at 12:34 AM, Anup Tiwari <an...@games24x7.com>
wrote:
> Hi All,
>
> We want to see all types of queries which ran on drill cluster or currently
> running from drill prompt, Can someone help us on this?
>
> To achieve above , we read the drill documentation and set up a storage
> plugin to access local file system and able to query
> *"drillbit_queries.json"* log file, but in above file we are getting status
> of all queries whose status is either "cancelled","completed" or "failed"
> but missing "running". At the same time we check drill profile interface
> where we can see running queries.
>
> I am sure if we can see on User-Interface then it must be coming from
> somewhere.
>
> Kindly help me on this.
>
> Regards,
> *Anup Tiwari*
> Software Engineer(BI-Team),PlayGames24x7 Pvt Ltd
>