You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@tez.apache.org by "Natarajan, Prabakaran 1. (NSN - IN/Bangalore)" <pr...@nsn.com> on 2014/05/20 08:59:23 UTC

Tez - Hive Parallel queries

Hi,

We tried running tez queries on Hive server 1 with 10 threads in Parallel. First 10 queries go through fine and result comes in 16-18 seconds but when we fire the same queries again on same connection we find that connection/queries are blocked for more than 200 seconds and only after some 200-260 seconds the queries get fired. In YARN job tracker UI we see the previous query containers still not being closed  and once they are closed after > 200 seconds the new queries get fired . We tried both enabling tez AM reuse( same JVM size for map and reduce ) and 100 ms value for  "tez.am.container.session.delay-allocation-millis" parameter but none of these help. Any idea on how can we reduce this > 200 sec block/latency

Thanks and Regards
Prabakaran.N  aka NP
nsn, Bangalore
When "I" is replaced by "We" - even Illness becomes "Wellness"





Re: Tez - Hive Parallel queries

Posted by Vikram Dixit <vi...@hortonworks.com>.
Hi Yogesh,

Hive server 1 is not currently supported with tez. It is unknown territory
and given that there is no active development/maintenance on hive server 1,
it is unlikely to be supported any time. Any reason for not moving to Hive
Server 2?

Thanks
Vikram.


On Tue, May 20, 2014 at 10:12 AM, Bikas Saha <bi...@hortonworks.com> wrote:

> Hi,
>
>
>
> Thanks for trying out Hive Server with Tez. The second set of queries
> should start executing on the same sessions/containers as the first query
> as soon as the previous queries are complete.
>
>
>
> We will need some information to understand whats happening.
>
> For starters
>
> 1)      Hive server has 10 threads in parallel right?
>
> 2)      How many queues and sessions per queue have been configured for
> Hive server?
>
> 3)      Have these queues been mapped to corresponding queues in YARN?
>
> 4)      What you are seeing is the following. You see full concurrency of
> 10 queries in the first set of queries. Then the second set of queries has
> gotten stalled for longer than expected. Right?
>
>
>
> I am adding Vikram, who is the implementer of Hive server support on Tez
> to this thread for more insights/questions.
>
>
>
> In the meanwhile, the best progress on this can be made by opening a jira
> in the Apache Hive project detailing your observations. Please attach your
> hive-site.xml and tez-site.xml to that jira. Its not possible to attach
> anything to this email thread. If you can share those files in any other
> manner and provide a link in this thread then it will help in the
> investigation.
>
>
>
> Thanks
>
> Bikas
>
>
>
>
>
> *From:* Natarajan, Prabakaran 1. (NSN - IN/Bangalore) [mailto:
> prabakaran.1.natarajan@nsn.com]
> *Sent:* Tuesday, May 20, 2014 1:22 AM
> *To:* user@tez.incubator.apache.org
> *Cc:* Sachwani, Yogesh (NSN - IN/Bangalore)
> *Subject:* RE: Tez - Hive Parallel queries
>
>
>
> Hi Rajesh
>
>
>
> We have this set in tez-site.xml and also before running the JDBC sql
> query we first fire a sql setting this parameter too.
>
>
>
> *Thanks and Regards*
>
> Prabakaran.N  aka NP
>
> nsn, Bangalore
>
> *When "I" is replaced by "We" - even Illness becomes "Wellness"*
>
>
>
>
>
> *From:* ext Rajesh Balamohan [mailto:rajesh.balamohan@gmail.com<ra...@gmail.com>]
>
> *Sent:* Tuesday, May 20, 2014 12:58 PM
> *To:* user@tez.incubator.apache.org
> *Cc:* Sachwani, Yogesh (NSN - IN/Bangalore)
> *Subject:* Re: Tez - Hive Parallel queries
>
>
>
> Hi Prbakaran,
>
>
>
> Are you passing these parameters via CLI or via tez-site.xml?
>
>
>
> ~Rajesh.B
>
>
>
> On Tue, May 20, 2014 at 12:29 PM, Natarajan, Prabakaran 1. (NSN -
> IN/Bangalore) <pr...@nsn.com> wrote:
>
> Hi,
>
>
>
> We tried running tez queries on Hive server 1 with 10 threads in Parallel.
> First 10 queries go through fine and result comes in 16-18 seconds but when
> we fire the same queries again on same connection we find that
> connection/queries are blocked for more than 200 seconds and only after
> some 200-260 seconds the queries get fired. In YARN job tracker UI we see
> the previous query containers still not being closed  and once they are
> closed after > 200 seconds the new queries get fired . We tried both
> enabling tez AM reuse( same JVM size for map and reduce ) and 100 ms value
> for  "tez.am.container.session.delay-allocation-millis" parameter but none
> of these help. Any idea on how can we reduce this > 200 sec block/latency
>
>
>
> *Thanks and Regards*
>
> Prabakaran.N  aka NP
>
> nsn, Bangalore
>
> *When "I" is replaced by "We" - even Illness becomes "Wellness"*
>
>
>
>
>
>
>
>
>
>
>
>
>
> --
> ~Rajesh.B
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

RE: Tez - Hive Parallel queries

Posted by Bikas Saha <bi...@hortonworks.com>.
Hi,



Thanks for trying out Hive Server with Tez. The second set of queries
should start executing on the same sessions/containers as the first query
as soon as the previous queries are complete.



We will need some information to understand whats happening.

For starters

1)      Hive server has 10 threads in parallel right?

2)      How many queues and sessions per queue have been configured for
Hive server?

3)      Have these queues been mapped to corresponding queues in YARN?

4)      What you are seeing is the following. You see full concurrency of
10 queries in the first set of queries. Then the second set of queries has
gotten stalled for longer than expected. Right?



I am adding Vikram, who is the implementer of Hive server support on Tez to
this thread for more insights/questions.



In the meanwhile, the best progress on this can be made by opening a jira
in the Apache Hive project detailing your observations. Please attach your
hive-site.xml and tez-site.xml to that jira. Its not possible to attach
anything to this email thread. If you can share those files in any other
manner and provide a link in this thread then it will help in the
investigation.



Thanks

Bikas





*From:* Natarajan, Prabakaran 1. (NSN - IN/Bangalore) [mailto:
prabakaran.1.natarajan@nsn.com]
*Sent:* Tuesday, May 20, 2014 1:22 AM
*To:* user@tez.incubator.apache.org
*Cc:* Sachwani, Yogesh (NSN - IN/Bangalore)
*Subject:* RE: Tez - Hive Parallel queries



Hi Rajesh



We have this set in tez-site.xml and also before running the JDBC sql query
we first fire a sql setting this parameter too.



*Thanks and Regards*

Prabakaran.N  aka NP

nsn, Bangalore

*When "I" is replaced by "We" - even Illness becomes "Wellness"*





*From:* ext Rajesh Balamohan
[mailto:rajesh.balamohan@gmail.com<ra...@gmail.com>]

*Sent:* Tuesday, May 20, 2014 12:58 PM
*To:* user@tez.incubator.apache.org
*Cc:* Sachwani, Yogesh (NSN - IN/Bangalore)
*Subject:* Re: Tez - Hive Parallel queries



Hi Prbakaran,



Are you passing these parameters via CLI or via tez-site.xml?



~Rajesh.B



On Tue, May 20, 2014 at 12:29 PM, Natarajan, Prabakaran 1. (NSN -
IN/Bangalore) <pr...@nsn.com> wrote:

Hi,



We tried running tez queries on Hive server 1 with 10 threads in Parallel.
First 10 queries go through fine and result comes in 16-18 seconds but when
we fire the same queries again on same connection we find that
connection/queries are blocked for more than 200 seconds and only after
some 200-260 seconds the queries get fired. In YARN job tracker UI we see
the previous query containers still not being closed  and once they are
closed after > 200 seconds the new queries get fired . We tried both
enabling tez AM reuse( same JVM size for map and reduce ) and 100 ms value
for  "tez.am.container.session.delay-allocation-millis" parameter but none
of these help. Any idea on how can we reduce this > 200 sec block/latency



*Thanks and Regards*

Prabakaran.N  aka NP

nsn, Bangalore

*When "I" is replaced by "We" - even Illness becomes "Wellness"*













-- 
~Rajesh.B

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

RE: Tez - Hive Parallel queries

Posted by "Natarajan, Prabakaran 1. (NSN - IN/Bangalore)" <pr...@nsn.com>.
Hi Rajesh

We have this set in tez-site.xml and also before running the JDBC sql query we first fire a sql setting this parameter too.

Thanks and Regards
Prabakaran.N  aka NP
nsn, Bangalore
When "I" is replaced by "We" - even Illness becomes "Wellness"


From: ext Rajesh Balamohan [mailto:rajesh.balamohan@gmail.com]
Sent: Tuesday, May 20, 2014 12:58 PM
To: user@tez.incubator.apache.org
Cc: Sachwani, Yogesh (NSN - IN/Bangalore)
Subject: Re: Tez - Hive Parallel queries

Hi Prbakaran,

Are you passing these parameters via CLI or via tez-site.xml?

~Rajesh.B

On Tue, May 20, 2014 at 12:29 PM, Natarajan, Prabakaran 1. (NSN - IN/Bangalore) <pr...@nsn.com>> wrote:
Hi,

We tried running tez queries on Hive server 1 with 10 threads in Parallel. First 10 queries go through fine and result comes in 16-18 seconds but when we fire the same queries again on same connection we find that connection/queries are blocked for more than 200 seconds and only after some 200-260 seconds the queries get fired. In YARN job tracker UI we see the previous query containers still not being closed  and once they are closed after > 200 seconds the new queries get fired . We tried both enabling tez AM reuse( same JVM size for map and reduce ) and 100 ms value for  "tez.am.container.session.delay-allocation-millis" parameter but none of these help. Any idea on how can we reduce this > 200 sec block/latency

Thanks and Regards
Prabakaran.N  aka NP
nsn, Bangalore
When "I" is replaced by "We" - even Illness becomes "Wellness"







--
~Rajesh.B

Re: Tez - Hive Parallel queries

Posted by Rajesh Balamohan <ra...@gmail.com>.
Hi Prbakaran,

Are you passing these parameters via CLI or via tez-site.xml?

~Rajesh.B


On Tue, May 20, 2014 at 12:29 PM, Natarajan, Prabakaran 1. (NSN -
IN/Bangalore) <pr...@nsn.com> wrote:

>  Hi,
>
> We tried running tez queries on Hive server 1 with 10 threads in Parallel.
> First 10 queries go through fine and result comes in 16-18 seconds but when
> we fire the same queries again on same connection we find that
> connection/queries are blocked for more than 200 seconds and only after
> some 200-260 seconds the queries get fired. In YARN job tracker UI we see
> the previous query containers still not being closed  and once they are
> closed after > 200 seconds the new queries get fired . We tried both
> enabling tez AM reuse( same JVM size for map and reduce ) and 100 ms value
> for  "tez.am.container.session.delay-allocation-millis" parameter but none
> of these help. Any idea on how can we reduce this > 200 sec block/latency
>
> *Thanks and Regards*
> Prabakaran.N  aka NP
> nsn, Bangalore
> *When "I" is replaced by "We" - even Illness becomes "Wellness"*
>
>
>
>
>



-- 
~Rajesh.B