You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Prasanth Jayachandran (Jira)" <ji...@apache.org> on 2020/05/11 19:57:00 UTC
[jira] [Created] (HIVE-23443) LLAP speculative task pre-emption
seems to be not working
Prasanth Jayachandran created HIVE-23443:
--------------------------------------------
Summary: LLAP speculative task pre-emption seems to be not working
Key: HIVE-23443
URL: https://issues.apache.org/jira/browse/HIVE-23443
Project: Hive
Issue Type: Bug
Reporter: Prasanth Jayachandran
I think after HIVE-23210 we are getting a stable sort order in pre-emption queue and it is causing pre-emption to not work in certain cases.
{code:java}
"attempt_1589167813851_0000_119_01_000008_0 (hive_20200511055921_89598f09-19f1-4969-ab7a-82e2dd796273-119/Map 1, started at 2020-05-11 05:59:22, in preemption queue, can finish)",
"attempt_1589167813851_0008_84_01_000008_1 (hive_20200511055928_7ae29ca3-e67d-4d1f-b193-05651023b503-84/Map 1, started at 2020-05-11 06:00:23, in preemption queue, can finish)" {code}
Scheduler only peek's at the pre-emption queue and looks at whether it is non-finishable.
[https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/TaskExecutorService.java#L420]
In the above case, all tasks are speculative but state change is not triggering pre-emption queue re-ordering so peek() always returns canFinish task even though non-finishable tasks are in the queue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)