You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@asterixdb.apache.org by sc...@informatik.hu-berlin.de on 2015/10/28 17:22:42 UTC
AsterixDB - query status / system status
Hello,
by now I was able to successfully install AsterixDB from the latest master
branch which also made it possible for me to load external data files into
the system. Thanks a lot for the help!
I am currently trying to rewrite some of the TPCH Queries to run them as
examples on AsterixDB. So far I was able to run some of the queries using
the web client, but others do not work yet. Unfortunatly sometimes I do
not get any error messages or results. On execution the output area of the
webclient turns blank as it usually does, but after some time I should get
the results shown (or an error). As I cannot tell if the queries are still
running, or if an error occured that is just not displayed, is there a
possibility to monitor the state of the system/query execution?
>From time to time the system even crashes and I cannot even shut it down
using managix stop.
Regards, Max
> Your assumption is correct, for the latest AsterixDB master we usually
> depend on the development version of Hyracks
> (https://github.com/apache/incubator-asterixdb-hyracks/). 'mvn install
> -DskipTests' with Hyracks should do the trick.
>
> Thanks,
> -Ian
>
Re: AsterixDB - query status / system status
Posted by Yingyi Bu <bu...@gmail.com>.
The ones I know are in our regression tests:
https://github.com/apache/incubator-asterixdb/tree/master/asterix-app/src/test/resources/runtimets/queries/tpch-sql-like/
But they are not the original ones --- we changed the filter conditions in
some queries to make sure their result set is not empty.
I guess Pouria may have more accurate ones.
Best,
Yingyi
On Wed, Oct 28, 2015 at 9:53 AM, Michael Carey <mj...@ics.uci.edu> wrote:
> Pouria indeed has a full set of TPC-H queries in AQL....
> (Yingyi does too, I believe.)
>
> On 10/28/15 9:49 AM, Ian Maxon wrote:
>
> Hi Max,
>
> Let me respond inline...
>
> I am currently trying to rewrite some of the TPCH Queries to run them as
> examples on AsterixDB. So far I was able to run some of the queries using
> the web client, but others do not work yet. Unfortunatly sometimes I do
> not get any error messages or results.
>
> I agree, the WebUI leaves something to be desired, especially for
> queries that take a long time.
> There is one minor amelioration to this however, that I find useful,
> the Hyracks admin console.
> It's on port 8888 on the CC at /adminconsole . It will show what jobs
> are running and which
> NCs are still registered with the CC.
>
> As for rewriting the TPC-H queries, I'm pretty sure we have these (but
> the ones in the tests folder aren't, they're toned-down versions). So
> maybe you don't have to rewrite anything, hopefully?
> @ Pouria, do you happen to have them handy?
>
> In general though, for more serious use, the HTTP api may be a better
> option, as it is more amenable to scripting.
>
>
> As I cannot tell if the queries are still
> running, or if an error occured that is just not displayed, is there a
> possibility to monitor the state of the system/query execution?
>
> The adminconsole will show if there's a job still running, but not
> exceptional states that cause the job to hang.
> The exceptions should be in the CC log always. Where that is depends
> on your managix configuration (but it's always called cc.log).
>
> >From time to time the system even crashes and I cannot even shut it down
>
> using managix stop.
>
> Yes, unfortunately 'managix stop' just goes through the list of NC's
> and requests 'kill' on the process, not 'kill' and then 'kill -9'. In
> the cases where an NC fails to exit politely, it has to be done by
> hand for managix.
>
> By the way, what environment are you running everything on? Is it a cluster?
>
> Thanks,
> -Ian
>
> On Wed, Oct 28, 2015 at 9:22 AM, <sc...@informatik.hu-berlin.de> <sc...@informatik.hu-berlin.de> wrote:
>
> Hello,
>
> by now I was able to successfully install AsterixDB from the latest master
> branch which also made it possible for me to load external data files into
> the system. Thanks a lot for the help!
>
> I am currently trying to rewrite some of the TPCH Queries to run them as
> examples on AsterixDB. So far I was able to run some of the queries using
> the web client, but others do not work yet. Unfortunatly sometimes I do
> not get any error messages or results. On execution the output area of the
> webclient turns blank as it usually does, but after some time I should get
> the results shown (or an error). As I cannot tell if the queries are still
> running, or if an error occured that is just not displayed, is there a
> possibility to monitor the state of the system/query execution?
>
> From time to time the system even crashes and I cannot even shut it down
> using managix stop.
>
> Regards, Max
>
>
>
> Your assumption is correct, for the latest AsterixDB master we usually
> depend on the development version of Hyracks
> (https://github.com/apache/incubator-asterixdb-hyracks/). 'mvn install
> -DskipTests' with Hyracks should do the trick.
>
> Thanks,
> -Ian
>
>
>
>
Re: AsterixDB - query status / system status
Posted by Michael Carey <mj...@ics.uci.edu>.
Pouria indeed has a full set of TPC-H queries in AQL....
(Yingyi does too, I believe.)
On 10/28/15 9:49 AM, Ian Maxon wrote:
> Hi Max,
>
> Let me respond inline...
>> I am currently trying to rewrite some of the TPCH Queries to run them as
>> examples on AsterixDB. So far I was able to run some of the queries using
>> the web client, but others do not work yet. Unfortunatly sometimes I do
>> not get any error messages or results.
> I agree, the WebUI leaves something to be desired, especially for
> queries that take a long time.
> There is one minor amelioration to this however, that I find useful,
> the Hyracks admin console.
> It's on port 8888 on the CC at /adminconsole . It will show what jobs
> are running and which
> NCs are still registered with the CC.
>
> As for rewriting the TPC-H queries, I'm pretty sure we have these (but
> the ones in the tests folder aren't, they're toned-down versions). So
> maybe you don't have to rewrite anything, hopefully?
> @ Pouria, do you happen to have them handy?
>
> In general though, for more serious use, the HTTP api may be a better
> option, as it is more amenable to scripting.
>
>> As I cannot tell if the queries are still
>> running, or if an error occured that is just not displayed, is there a
>> possibility to monitor the state of the system/query execution?
> The adminconsole will show if there's a job still running, but not
> exceptional states that cause the job to hang.
> The exceptions should be in the CC log always. Where that is depends
> on your managix configuration (but it's always called cc.log).
>
> >From time to time the system even crashes and I cannot even shut it down
>> using managix stop.
> Yes, unfortunately 'managix stop' just goes through the list of NC's
> and requests 'kill' on the process, not 'kill' and then 'kill -9'. In
> the cases where an NC fails to exit politely, it has to be done by
> hand for managix.
>
> By the way, what environment are you running everything on? Is it a cluster?
>
> Thanks,
> -Ian
>
> On Wed, Oct 28, 2015 at 9:22 AM, <sc...@informatik.hu-berlin.de> wrote:
>> Hello,
>>
>> by now I was able to successfully install AsterixDB from the latest master
>> branch which also made it possible for me to load external data files into
>> the system. Thanks a lot for the help!
>>
>> I am currently trying to rewrite some of the TPCH Queries to run them as
>> examples on AsterixDB. So far I was able to run some of the queries using
>> the web client, but others do not work yet. Unfortunatly sometimes I do
>> not get any error messages or results. On execution the output area of the
>> webclient turns blank as it usually does, but after some time I should get
>> the results shown (or an error). As I cannot tell if the queries are still
>> running, or if an error occured that is just not displayed, is there a
>> possibility to monitor the state of the system/query execution?
>>
>> From time to time the system even crashes and I cannot even shut it down
>> using managix stop.
>>
>> Regards, Max
>>
>>
>>> Your assumption is correct, for the latest AsterixDB master we usually
>>> depend on the development version of Hyracks
>>> (https://github.com/apache/incubator-asterixdb-hyracks/). 'mvn install
>>> -DskipTests' with Hyracks should do the trick.
>>>
>>> Thanks,
>>> -Ian
>>>
Re: AsterixDB - query status / system status
Posted by Pouria Pirzadeh <po...@gmail.com>.
Yingyi is correct about the ones under regression tests. Some of them are
modified to work fine on tiny scales of data.
You can find the actual TPCH queries, in AQL, under:
https://github.com/apache/incubator-asterixdb/tree/master/asterix-benchmarks/src/main/resources/tpc-h/queries
Let me know if you need more help about them.
Pouria
On Wed, Oct 28, 2015 at 10:34 AM, <sc...@informatik.hu-berlin.de> wrote:
> Hi Ian,
>
> thanks a lot for your help! Currently I prepare some of those tests on my
> local computer. Starting on Monday I will have access to 8 nodes of a
> cluster for another week.
>
> Regards, Max
>
> > Hi Max,
> >
> > Let me respond inline...
> >>I am currently trying to rewrite some of the TPCH Queries to run them as
> >>examples on AsterixDB. So far I was able to run some of the queries using
> >>the web client, but others do not work yet. Unfortunatly sometimes I do
> >>not get any error messages or results.
> >
> > I agree, the WebUI leaves something to be desired, especially for
> > queries that take a long time.
> > There is one minor amelioration to this however, that I find useful,
> > the Hyracks admin console.
> > It's on port 8888 on the CC at /adminconsole . It will show what jobs
> > are running and which
> > NCs are still registered with the CC.
> >
> > As for rewriting the TPC-H queries, I'm pretty sure we have these (but
> > the ones in the tests folder aren't, they're toned-down versions). So
> > maybe you don't have to rewrite anything, hopefully?
> > @ Pouria, do you happen to have them handy?
> >
> > In general though, for more serious use, the HTTP api may be a better
> > option, as it is more amenable to scripting.
> >
> >>As I cannot tell if the queries are still
> >>running, or if an error occured that is just not displayed, is there a
> >>possibility to monitor the state of the system/query execution?
> >
> > The adminconsole will show if there's a job still running, but not
> > exceptional states that cause the job to hang.
> > The exceptions should be in the CC log always. Where that is depends
> > on your managix configuration (but it's always called cc.log).
> >
> >>From time to time the system even crashes and I cannot even shut it down
> >>using managix stop.
> >
> > Yes, unfortunately 'managix stop' just goes through the list of NC's
> > and requests 'kill' on the process, not 'kill' and then 'kill -9'. In
> > the cases where an NC fails to exit politely, it has to be done by
> > hand for managix.
> >
> > By the way, what environment are you running everything on? Is it a
> > cluster?
> >
> > Thanks,
> > -Ian
> >
> > On Wed, Oct 28, 2015 at 9:22 AM, <sc...@informatik.hu-berlin.de>
> > wrote:
> >> Hello,
> >>
> >> by now I was able to successfully install AsterixDB from the latest
> >> master
> >> branch which also made it possible for me to load external data files
> >> into
> >> the system. Thanks a lot for the help!
> >>
> >> I am currently trying to rewrite some of the TPCH Queries to run them as
> >> examples on AsterixDB. So far I was able to run some of the queries
> >> using
> >> the web client, but others do not work yet. Unfortunatly sometimes I do
> >> not get any error messages or results. On execution the output area of
> >> the
> >> webclient turns blank as it usually does, but after some time I should
> >> get
> >> the results shown (or an error). As I cannot tell if the queries are
> >> still
> >> running, or if an error occured that is just not displayed, is there a
> >> possibility to monitor the state of the system/query execution?
> >>
> >> From time to time the system even crashes and I cannot even shut it down
> >> using managix stop.
> >>
> >> Regards, Max
> >>
> >>
> >>> Your assumption is correct, for the latest AsterixDB master we usually
> >>> depend on the development version of Hyracks
> >>> (https://github.com/apache/incubator-asterixdb-hyracks/). 'mvn install
> >>> -DskipTests' with Hyracks should do the trick.
> >>>
> >>> Thanks,
> >>> -Ian
> >>>
> >>
> >
>
>
>
Re: AsterixDB - query status / system status
Posted by sc...@informatik.hu-berlin.de.
Hi Ian,
thanks a lot for your help! Currently I prepare some of those tests on my
local computer. Starting on Monday I will have access to 8 nodes of a
cluster for another week.
Regards, Max
> Hi Max,
>
> Let me respond inline...
>>I am currently trying to rewrite some of the TPCH Queries to run them as
>>examples on AsterixDB. So far I was able to run some of the queries using
>>the web client, but others do not work yet. Unfortunatly sometimes I do
>>not get any error messages or results.
>
> I agree, the WebUI leaves something to be desired, especially for
> queries that take a long time.
> There is one minor amelioration to this however, that I find useful,
> the Hyracks admin console.
> It's on port 8888 on the CC at /adminconsole . It will show what jobs
> are running and which
> NCs are still registered with the CC.
>
> As for rewriting the TPC-H queries, I'm pretty sure we have these (but
> the ones in the tests folder aren't, they're toned-down versions). So
> maybe you don't have to rewrite anything, hopefully?
> @ Pouria, do you happen to have them handy?
>
> In general though, for more serious use, the HTTP api may be a better
> option, as it is more amenable to scripting.
>
>>As I cannot tell if the queries are still
>>running, or if an error occured that is just not displayed, is there a
>>possibility to monitor the state of the system/query execution?
>
> The adminconsole will show if there's a job still running, but not
> exceptional states that cause the job to hang.
> The exceptions should be in the CC log always. Where that is depends
> on your managix configuration (but it's always called cc.log).
>
>>From time to time the system even crashes and I cannot even shut it down
>>using managix stop.
>
> Yes, unfortunately 'managix stop' just goes through the list of NC's
> and requests 'kill' on the process, not 'kill' and then 'kill -9'. In
> the cases where an NC fails to exit politely, it has to be done by
> hand for managix.
>
> By the way, what environment are you running everything on? Is it a
> cluster?
>
> Thanks,
> -Ian
>
> On Wed, Oct 28, 2015 at 9:22 AM, <sc...@informatik.hu-berlin.de>
> wrote:
>> Hello,
>>
>> by now I was able to successfully install AsterixDB from the latest
>> master
>> branch which also made it possible for me to load external data files
>> into
>> the system. Thanks a lot for the help!
>>
>> I am currently trying to rewrite some of the TPCH Queries to run them as
>> examples on AsterixDB. So far I was able to run some of the queries
>> using
>> the web client, but others do not work yet. Unfortunatly sometimes I do
>> not get any error messages or results. On execution the output area of
>> the
>> webclient turns blank as it usually does, but after some time I should
>> get
>> the results shown (or an error). As I cannot tell if the queries are
>> still
>> running, or if an error occured that is just not displayed, is there a
>> possibility to monitor the state of the system/query execution?
>>
>> From time to time the system even crashes and I cannot even shut it down
>> using managix stop.
>>
>> Regards, Max
>>
>>
>>> Your assumption is correct, for the latest AsterixDB master we usually
>>> depend on the development version of Hyracks
>>> (https://github.com/apache/incubator-asterixdb-hyracks/). 'mvn install
>>> -DskipTests' with Hyracks should do the trick.
>>>
>>> Thanks,
>>> -Ian
>>>
>>
>
Re: AsterixDB - query status / system status
Posted by Ian Maxon <im...@uci.edu>.
Hi Max,
Let me respond inline...
>I am currently trying to rewrite some of the TPCH Queries to run them as
>examples on AsterixDB. So far I was able to run some of the queries using
>the web client, but others do not work yet. Unfortunatly sometimes I do
>not get any error messages or results.
I agree, the WebUI leaves something to be desired, especially for
queries that take a long time.
There is one minor amelioration to this however, that I find useful,
the Hyracks admin console.
It's on port 8888 on the CC at /adminconsole . It will show what jobs
are running and which
NCs are still registered with the CC.
As for rewriting the TPC-H queries, I'm pretty sure we have these (but
the ones in the tests folder aren't, they're toned-down versions). So
maybe you don't have to rewrite anything, hopefully?
@ Pouria, do you happen to have them handy?
In general though, for more serious use, the HTTP api may be a better
option, as it is more amenable to scripting.
>As I cannot tell if the queries are still
>running, or if an error occured that is just not displayed, is there a
>possibility to monitor the state of the system/query execution?
The adminconsole will show if there's a job still running, but not
exceptional states that cause the job to hang.
The exceptions should be in the CC log always. Where that is depends
on your managix configuration (but it's always called cc.log).
>From time to time the system even crashes and I cannot even shut it down
>using managix stop.
Yes, unfortunately 'managix stop' just goes through the list of NC's
and requests 'kill' on the process, not 'kill' and then 'kill -9'. In
the cases where an NC fails to exit politely, it has to be done by
hand for managix.
By the way, what environment are you running everything on? Is it a cluster?
Thanks,
-Ian
On Wed, Oct 28, 2015 at 9:22 AM, <sc...@informatik.hu-berlin.de> wrote:
> Hello,
>
> by now I was able to successfully install AsterixDB from the latest master
> branch which also made it possible for me to load external data files into
> the system. Thanks a lot for the help!
>
> I am currently trying to rewrite some of the TPCH Queries to run them as
> examples on AsterixDB. So far I was able to run some of the queries using
> the web client, but others do not work yet. Unfortunatly sometimes I do
> not get any error messages or results. On execution the output area of the
> webclient turns blank as it usually does, but after some time I should get
> the results shown (or an error). As I cannot tell if the queries are still
> running, or if an error occured that is just not displayed, is there a
> possibility to monitor the state of the system/query execution?
>
> From time to time the system even crashes and I cannot even shut it down
> using managix stop.
>
> Regards, Max
>
>
>> Your assumption is correct, for the latest AsterixDB master we usually
>> depend on the development version of Hyracks
>> (https://github.com/apache/incubator-asterixdb-hyracks/). 'mvn install
>> -DskipTests' with Hyracks should do the trick.
>>
>> Thanks,
>> -Ian
>>
>