You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "László Bodor (Jira)" <ji...@apache.org> on 2023/04/21 07:27:00 UTC
[jira] [Updated] (HIVE-27283) Use tez.local.mode in HiveServer2 for trivial queries
[ https://issues.apache.org/jira/browse/HIVE-27283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
László Bodor updated HIVE-27283:
--------------------------------
Description:
Today, a query like this:
{code}
INSERT INTO TABLE students VALUES ('fred flintstone', 35, 1.28), ('barney rubble', 32, 2.32);
{code}
spins up a TezAM and containers. I believe this is not optimal, even if we already have an tez application running. Not to mention setups where only a hiveserver2 is alive and TezAMs + LLAP executors are spun up on demand.
A possible risk is to overwhelm Hiveserver2 with such queries, this scenario should be handled with care.
My proposal is to maintain a local tez session pool (default size 0, recommended is 1...4) in hs2, and in compile-time let's identify "trivial queries" that by default needs tez application (like the INSERT INTO above).
The first implementation can include only simply INSERT INTO queries, and we can decide the rest later.
> Use tez.local.mode in HiveServer2 for trivial queries
> -----------------------------------------------------
>
> Key: HIVE-27283
> URL: https://issues.apache.org/jira/browse/HIVE-27283
> Project: Hive
> Issue Type: Improvement
> Reporter: László Bodor
> Priority: Major
>
> Today, a query like this:
> {code}
> INSERT INTO TABLE students VALUES ('fred flintstone', 35, 1.28), ('barney rubble', 32, 2.32);
> {code}
> spins up a TezAM and containers. I believe this is not optimal, even if we already have an tez application running. Not to mention setups where only a hiveserver2 is alive and TezAMs + LLAP executors are spun up on demand.
> A possible risk is to overwhelm Hiveserver2 with such queries, this scenario should be handled with care.
> My proposal is to maintain a local tez session pool (default size 0, recommended is 1...4) in hs2, and in compile-time let's identify "trivial queries" that by default needs tez application (like the INSERT INTO above).
> The first implementation can include only simply INSERT INTO queries, and we can decide the rest later.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)