You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by George Lu <lu...@gmail.com> on 2015/05/04 09:46:57 UTC
Questions on drill execution
Dear all,
I have deployed Drill on several nodes in the cluster and experienced the
query.
I have some questions:
1) Can anyone show me some links to read for the Drill execution plan, how
to understand it.
2) I execute some group by query on HBase several times, some time very
fast, while sometimes very slow, and even get exception, like "Query
failed: RemoteRpcException: Failure while running fragment., Command failed
while establishing connection. Failure type CONNECTION. [
2ba2092f-53ed-480a-9f23-f89b6e6a91a0 on test6:31010 ]
[ 2ba2092f-53ed-480a-9f23-f89b6e6a91a0 on test6:31010 ]
java.lang.RuntimeException: java.sql.SQLException: Failure while executing
query.
at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
at sqlline.SqlLine.print(SqlLine.java:1809)
at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
at sqlline.SqlLine.dispatch(SqlLine.java:889)
at sqlline.SqlLine.begin(SqlLine.java:763)
at sqlline.SqlLine.start(SqlLine.java:498)
at sqlline.SqlLine.main(SqlLine.java:460)"
I am not sure whether Drill uses pre-execute like in Hadoop or auto failure
detection. If some of the execution node failed, what will happen? If some
node is under high pressure for other task, and what will happen if this
node becomes the one who drags the whole execution slow?
Thanks!
George
Re: Questions on drill execution
Posted by Ted Dunning <te...@gmail.com>.
Cool!
On Tue, May 5, 2015 at 3:39 AM, Tomer Shiran <ts...@gmail.com> wrote:
> http://drill.apache.org/docs/drill-introduction/#videos
>
>
>
> > On May 4, 2015, at 3:10 PM, Ted Dunning <te...@gmail.com> wrote:
> >
> > Is that video linked from the web site? Looks like a really great
> resource.
> >
> >
> >
> >> On Mon, May 4, 2015 at 6:23 PM, Hao Zhu <hz...@maprtech.com> wrote:
> >>
> >> https://www.youtube.com/watch?v=kG6vzsk8T7E
> >> This presentation contains lots of details for query execution plan.
> >>
> >> On Mon, May 4, 2015 at 7:10 AM, Carol McDonald <cm...@maprtech.com>
> >> wrote:
> >>
> >>> https://cwiki.apache.org/confluence/display/DRILL/EXPLAIN+Commands
> >>>
> >>>> On Mon, May 4, 2015 at 3:46 AM, George Lu <lu...@gmail.com>
> wrote:
> >>>>
> >>>> Dear all,
> >>>>
> >>>> I have deployed Drill on several nodes in the cluster and experienced
> >> the
> >>>> query.
> >>>> I have some questions:
> >>>>
> >>>> 1) Can anyone show me some links to read for the Drill execution plan,
> >>> how
> >>>> to understand it.
> >>>>
> >>>> 2) I execute some group by query on HBase several times, some time
> very
> >>>> fast, while sometimes very slow, and even get exception, like "Query
> >>>> failed: RemoteRpcException: Failure while running fragment., Command
> >>> failed
> >>>> while establishing connection. Failure type CONNECTION. [
> >>>> 2ba2092f-53ed-480a-9f23-f89b6e6a91a0 on test6:31010 ]
> >>>> [ 2ba2092f-53ed-480a-9f23-f89b6e6a91a0 on test6:31010 ]
> >>>>
> >>>>
> >>>> java.lang.RuntimeException: java.sql.SQLException: Failure while
> >>> executing
> >>>> query.
> >>>> at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
> >>>> at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
> >>>> at sqlline.SqlLine.print(SqlLine.java:1809)
> >>>> at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
> >>>> at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
> >>>> at sqlline.SqlLine.dispatch(SqlLine.java:889)
> >>>> at sqlline.SqlLine.begin(SqlLine.java:763)
> >>>> at sqlline.SqlLine.start(SqlLine.java:498)
> >>>> at sqlline.SqlLine.main(SqlLine.java:460)"
> >>>>
> >>>> I am not sure whether Drill uses pre-execute like in Hadoop or auto
> >>> failure
> >>>> detection. If some of the execution node failed, what will happen? If
> >>> some
> >>>> node is under high pressure for other task, and what will happen if
> >> this
> >>>> node becomes the one who drags the whole execution slow?
> >>>>
> >>>> Thanks!
> >>>>
> >>>> George
> >>
>
Re: Questions on drill execution
Posted by George Lu <lu...@gmail.com>.
Thanks all!
George Lu
On Tue, May 5, 2015 at 10:39 AM, Tomer Shiran <ts...@gmail.com> wrote:
> http://drill.apache.org/docs/drill-introduction/#videos
>
>
>
> > On May 4, 2015, at 3:10 PM, Ted Dunning <te...@gmail.com> wrote:
> >
> > Is that video linked from the web site? Looks like a really great
> resource.
> >
> >
> >
> >> On Mon, May 4, 2015 at 6:23 PM, Hao Zhu <hz...@maprtech.com> wrote:
> >>
> >> https://www.youtube.com/watch?v=kG6vzsk8T7E
> >> This presentation contains lots of details for query execution plan.
> >>
> >> On Mon, May 4, 2015 at 7:10 AM, Carol McDonald <cm...@maprtech.com>
> >> wrote:
> >>
> >>> https://cwiki.apache.org/confluence/display/DRILL/EXPLAIN+Commands
> >>>
> >>>> On Mon, May 4, 2015 at 3:46 AM, George Lu <lu...@gmail.com>
> wrote:
> >>>>
> >>>> Dear all,
> >>>>
> >>>> I have deployed Drill on several nodes in the cluster and experienced
> >> the
> >>>> query.
> >>>> I have some questions:
> >>>>
> >>>> 1) Can anyone show me some links to read for the Drill execution plan,
> >>> how
> >>>> to understand it.
> >>>>
> >>>> 2) I execute some group by query on HBase several times, some time
> very
> >>>> fast, while sometimes very slow, and even get exception, like "Query
> >>>> failed: RemoteRpcException: Failure while running fragment., Command
> >>> failed
> >>>> while establishing connection. Failure type CONNECTION. [
> >>>> 2ba2092f-53ed-480a-9f23-f89b6e6a91a0 on test6:31010 ]
> >>>> [ 2ba2092f-53ed-480a-9f23-f89b6e6a91a0 on test6:31010 ]
> >>>>
> >>>>
> >>>> java.lang.RuntimeException: java.sql.SQLException: Failure while
> >>> executing
> >>>> query.
> >>>> at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
> >>>> at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
> >>>> at sqlline.SqlLine.print(SqlLine.java:1809)
> >>>> at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
> >>>> at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
> >>>> at sqlline.SqlLine.dispatch(SqlLine.java:889)
> >>>> at sqlline.SqlLine.begin(SqlLine.java:763)
> >>>> at sqlline.SqlLine.start(SqlLine.java:498)
> >>>> at sqlline.SqlLine.main(SqlLine.java:460)"
> >>>>
> >>>> I am not sure whether Drill uses pre-execute like in Hadoop or auto
> >>> failure
> >>>> detection. If some of the execution node failed, what will happen? If
> >>> some
> >>>> node is under high pressure for other task, and what will happen if
> >> this
> >>>> node becomes the one who drags the whole execution slow?
> >>>>
> >>>> Thanks!
> >>>>
> >>>> George
> >>
>
Re: Questions on drill execution
Posted by Tomer Shiran <ts...@gmail.com>.
http://drill.apache.org/docs/drill-introduction/#videos
> On May 4, 2015, at 3:10 PM, Ted Dunning <te...@gmail.com> wrote:
>
> Is that video linked from the web site? Looks like a really great resource.
>
>
>
>> On Mon, May 4, 2015 at 6:23 PM, Hao Zhu <hz...@maprtech.com> wrote:
>>
>> https://www.youtube.com/watch?v=kG6vzsk8T7E
>> This presentation contains lots of details for query execution plan.
>>
>> On Mon, May 4, 2015 at 7:10 AM, Carol McDonald <cm...@maprtech.com>
>> wrote:
>>
>>> https://cwiki.apache.org/confluence/display/DRILL/EXPLAIN+Commands
>>>
>>>> On Mon, May 4, 2015 at 3:46 AM, George Lu <lu...@gmail.com> wrote:
>>>>
>>>> Dear all,
>>>>
>>>> I have deployed Drill on several nodes in the cluster and experienced
>> the
>>>> query.
>>>> I have some questions:
>>>>
>>>> 1) Can anyone show me some links to read for the Drill execution plan,
>>> how
>>>> to understand it.
>>>>
>>>> 2) I execute some group by query on HBase several times, some time very
>>>> fast, while sometimes very slow, and even get exception, like "Query
>>>> failed: RemoteRpcException: Failure while running fragment., Command
>>> failed
>>>> while establishing connection. Failure type CONNECTION. [
>>>> 2ba2092f-53ed-480a-9f23-f89b6e6a91a0 on test6:31010 ]
>>>> [ 2ba2092f-53ed-480a-9f23-f89b6e6a91a0 on test6:31010 ]
>>>>
>>>>
>>>> java.lang.RuntimeException: java.sql.SQLException: Failure while
>>> executing
>>>> query.
>>>> at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
>>>> at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
>>>> at sqlline.SqlLine.print(SqlLine.java:1809)
>>>> at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
>>>> at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
>>>> at sqlline.SqlLine.dispatch(SqlLine.java:889)
>>>> at sqlline.SqlLine.begin(SqlLine.java:763)
>>>> at sqlline.SqlLine.start(SqlLine.java:498)
>>>> at sqlline.SqlLine.main(SqlLine.java:460)"
>>>>
>>>> I am not sure whether Drill uses pre-execute like in Hadoop or auto
>>> failure
>>>> detection. If some of the execution node failed, what will happen? If
>>> some
>>>> node is under high pressure for other task, and what will happen if
>> this
>>>> node becomes the one who drags the whole execution slow?
>>>>
>>>> Thanks!
>>>>
>>>> George
>>
Re: Questions on drill execution
Posted by Ted Dunning <te...@gmail.com>.
Is that video linked from the web site? Looks like a really great resource.
On Mon, May 4, 2015 at 6:23 PM, Hao Zhu <hz...@maprtech.com> wrote:
> https://www.youtube.com/watch?v=kG6vzsk8T7E
> This presentation contains lots of details for query execution plan.
>
> On Mon, May 4, 2015 at 7:10 AM, Carol McDonald <cm...@maprtech.com>
> wrote:
>
> > https://cwiki.apache.org/confluence/display/DRILL/EXPLAIN+Commands
> >
> > On Mon, May 4, 2015 at 3:46 AM, George Lu <lu...@gmail.com> wrote:
> >
> > > Dear all,
> > >
> > > I have deployed Drill on several nodes in the cluster and experienced
> the
> > > query.
> > > I have some questions:
> > >
> > > 1) Can anyone show me some links to read for the Drill execution plan,
> > how
> > > to understand it.
> > >
> > > 2) I execute some group by query on HBase several times, some time very
> > > fast, while sometimes very slow, and even get exception, like "Query
> > > failed: RemoteRpcException: Failure while running fragment., Command
> > failed
> > > while establishing connection. Failure type CONNECTION. [
> > > 2ba2092f-53ed-480a-9f23-f89b6e6a91a0 on test6:31010 ]
> > > [ 2ba2092f-53ed-480a-9f23-f89b6e6a91a0 on test6:31010 ]
> > >
> > >
> > > java.lang.RuntimeException: java.sql.SQLException: Failure while
> > executing
> > > query.
> > > at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
> > > at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
> > > at sqlline.SqlLine.print(SqlLine.java:1809)
> > > at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
> > > at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
> > > at sqlline.SqlLine.dispatch(SqlLine.java:889)
> > > at sqlline.SqlLine.begin(SqlLine.java:763)
> > > at sqlline.SqlLine.start(SqlLine.java:498)
> > > at sqlline.SqlLine.main(SqlLine.java:460)"
> > >
> > > I am not sure whether Drill uses pre-execute like in Hadoop or auto
> > failure
> > > detection. If some of the execution node failed, what will happen? If
> > some
> > > node is under high pressure for other task, and what will happen if
> this
> > > node becomes the one who drags the whole execution slow?
> > >
> > > Thanks!
> > >
> > > George
> > >
> >
>
Re: Questions on drill execution
Posted by Hao Zhu <hz...@maprtech.com>.
https://www.youtube.com/watch?v=kG6vzsk8T7E
This presentation contains lots of details for query execution plan.
On Mon, May 4, 2015 at 7:10 AM, Carol McDonald <cm...@maprtech.com>
wrote:
> https://cwiki.apache.org/confluence/display/DRILL/EXPLAIN+Commands
>
> On Mon, May 4, 2015 at 3:46 AM, George Lu <lu...@gmail.com> wrote:
>
> > Dear all,
> >
> > I have deployed Drill on several nodes in the cluster and experienced the
> > query.
> > I have some questions:
> >
> > 1) Can anyone show me some links to read for the Drill execution plan,
> how
> > to understand it.
> >
> > 2) I execute some group by query on HBase several times, some time very
> > fast, while sometimes very slow, and even get exception, like "Query
> > failed: RemoteRpcException: Failure while running fragment., Command
> failed
> > while establishing connection. Failure type CONNECTION. [
> > 2ba2092f-53ed-480a-9f23-f89b6e6a91a0 on test6:31010 ]
> > [ 2ba2092f-53ed-480a-9f23-f89b6e6a91a0 on test6:31010 ]
> >
> >
> > java.lang.RuntimeException: java.sql.SQLException: Failure while
> executing
> > query.
> > at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
> > at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
> > at sqlline.SqlLine.print(SqlLine.java:1809)
> > at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
> > at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
> > at sqlline.SqlLine.dispatch(SqlLine.java:889)
> > at sqlline.SqlLine.begin(SqlLine.java:763)
> > at sqlline.SqlLine.start(SqlLine.java:498)
> > at sqlline.SqlLine.main(SqlLine.java:460)"
> >
> > I am not sure whether Drill uses pre-execute like in Hadoop or auto
> failure
> > detection. If some of the execution node failed, what will happen? If
> some
> > node is under high pressure for other task, and what will happen if this
> > node becomes the one who drags the whole execution slow?
> >
> > Thanks!
> >
> > George
> >
>
Re: Questions on drill execution
Posted by Carol McDonald <cm...@maprtech.com>.
https://cwiki.apache.org/confluence/display/DRILL/EXPLAIN+Commands
On Mon, May 4, 2015 at 3:46 AM, George Lu <lu...@gmail.com> wrote:
> Dear all,
>
> I have deployed Drill on several nodes in the cluster and experienced the
> query.
> I have some questions:
>
> 1) Can anyone show me some links to read for the Drill execution plan, how
> to understand it.
>
> 2) I execute some group by query on HBase several times, some time very
> fast, while sometimes very slow, and even get exception, like "Query
> failed: RemoteRpcException: Failure while running fragment., Command failed
> while establishing connection. Failure type CONNECTION. [
> 2ba2092f-53ed-480a-9f23-f89b6e6a91a0 on test6:31010 ]
> [ 2ba2092f-53ed-480a-9f23-f89b6e6a91a0 on test6:31010 ]
>
>
> java.lang.RuntimeException: java.sql.SQLException: Failure while executing
> query.
> at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
> at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
> at sqlline.SqlLine.print(SqlLine.java:1809)
> at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
> at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
> at sqlline.SqlLine.dispatch(SqlLine.java:889)
> at sqlline.SqlLine.begin(SqlLine.java:763)
> at sqlline.SqlLine.start(SqlLine.java:498)
> at sqlline.SqlLine.main(SqlLine.java:460)"
>
> I am not sure whether Drill uses pre-execute like in Hadoop or auto failure
> detection. If some of the execution node failed, what will happen? If some
> node is under high pressure for other task, and what will happen if this
> node becomes the one who drags the whole execution slow?
>
> Thanks!
>
> George
>