How to obtain concurrent query executions

[Jose Rozanec <jo...@mercadolibre.com>]

Hi,

We have a Hive cluster. We notice that some queries consume all resources,
which is not desirable to us, since we want to grant some degree of
parallelism to incoming ones: any incoming query should be able to do at
least some progress, not just wait the big one finish.

Is there way to do so? We use Hive 2.1.0 with Tez engine.

Thank you in advance,

Joze.

Re: How to obtain concurrent query executions

[Jose Rozanec <jo...@mercadolibre.com>]

Thanks!

2016-09-28 12:55 GMT-03:00 Frank Luo <jl...@merkleinc.com>:

> If you are using Hadoop 2.7 or newer, you can use
> mapreduce.job.running.map.limit and mapreduce.job.running.reduce.limit to
> restrict map and reduce tasks at each job level.
>
> Another way is to use Scheduler to limit queue size.
>
>
>
> *From:* Jose Rozanec [mailto:jose.rozanec@mercadolibre.com]
> *Sent:* Tuesday, September 27, 2016 5:54 PM
> *To:* user@hive.apache.org
> *Subject:* How to obtain concurrent query executions
>
>
>
> Hi,
>
>
>
> We have a Hive cluster. We notice that some queries consume all resources,
> which is not desirable to us, since we want to grant some degree of
> parallelism to incoming ones: any incoming query should be able to do at
> least some progress, not just wait the big one finish.
>
>
>
> Is there way to do so? We use Hive 2.1.0 with Tez engine.
>
>
>
> Thank you in advance,
>
>
>
> Joze.
>
> *Access the Q2 2016 Digital Marketing Report for a fresh set of trends and
> benchmarks in digital marketing*
> <http://www2.merkleinc.com/l/47252/2016-07-26/47gt7c>
>
> *Download our latest report titled “The Case for Change: Exploring the
> Myths of Customer-Centric Transformation” *
> <http://www2.merkleinc.com/l/47252/2016-08-04/4b9p7c>
>
> This email and any attachments transmitted with it are intended for use by
> the intended recipient(s) only. If you have received this email in error,
> please notify the sender immediately and then delete it. If you are not the
> intended recipient, you must not keep, use, disclose, copy or distribute
> this email without the author’s prior permission. We take precautions to
> minimize the risk of transmitting software viruses, but we advise you to
> perform your own virus checks on any attachment to this message. We cannot
> accept liability for any loss or damage caused by software viruses. The
> information contained in this communication may be confidential and may be
> subject to the attorney-client privilege.
>

RE: How to obtain concurrent query executions

[Frank Luo <jl...@merkleinc.com>]

If you are using Hadoop 2.7 or newer, you can use mapreduce.job.running.map.limit and mapreduce.job.running.reduce.limit to restrict map and reduce tasks at each job level.

Another way is to use Scheduler to limit queue size.

From: Jose Rozanec [mailto:jose.rozanec@mercadolibre.com]
Sent: Tuesday, September 27, 2016 5:54 PM
To: user@hive.apache.org
Subject: How to obtain concurrent query executions

Hi,

We have a Hive cluster. We notice that some queries consume all resources, which is not desirable to us, since we want to grant some degree of parallelism to incoming ones: any incoming query should be able to do at least some progress, not just wait the big one finish.

Is there way to do so? We use Hive 2.1.0 with Tez engine.

Thank you in advance,

Joze.

Access the Q2 2016 Digital Marketing Report for a fresh set of trends and benchmarks in digital marketing<http://www2.merkleinc.com/l/47252/2016-07-26/47gt7c>

Download our latest report titled “The Case for Change: Exploring the Myths of Customer-Centric Transformation” <http://www2.merkleinc.com/l/47252/2016-08-04/4b9p7c>

This email and any attachments transmitted with it are intended for use by the intended recipient(s) only. If you have received this email in error, please notify the sender immediately and then delete it. If you are not the intended recipient, you must not keep, use, disclose, copy or distribute this email without the author’s prior permission. We take precautions to minimize the risk of transmitting software viruses, but we advise you to perform your own virus checks on any attachment to this message. We cannot accept liability for any loss or damage caused by software viruses. The information contained in this communication may be confidential and may be subject to the attorney-client privilege.