You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "Rajesh Balamohan (JIRA)" <ji...@apache.org> on 2015/03/26 11:33:53 UTC
[jira] [Comment Edited] (TEZ-2217) The min-held-containers
constraint is not enforced during query runtime
[ https://issues.apache.org/jira/browse/TEZ-2217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14381671#comment-14381671 ]
Rajesh Balamohan edited comment on TEZ-2217 at 3/26/15 10:33 AM:
-----------------------------------------------------------------
lgtm. + 1. Minor comments.
- YarnTaskSchedulerService line 648. log statement is repeated. May be it was intented to be debug log.
- YarnTaskSchedulerService line 569. expireTime can be reused instead of recomputing expireTimeMin.
- TestTaskScheduler Test doesn't cover the scenario when taskrequests are not empty. It should be fine, as the patch mainly updates the expiry time for the sessionMinHeldContainers so that they do not get released.
The issue was happening even with 20-40 second queries, but at a much lower pace. Verified that the patch holds min containers.
was (Author: rajesh.balamohan):
Minor comments.
- YarnTaskSchedulerService line 648. log statement is repeated. May be it was intented to be debug log.
- YarnTaskSchedulerService line 569. expireTime can be reused instead of recomputing expireTimeMin.
- TestTaskScheduler Test doesn't cover the scenario when taskrequests are not empty. It should be fine, as the patch mainly updates the expiry time for the sessionMinHeldContainers so that they do not get released.
The issue was happening even with 20-40 second queries, but at a much lower pace. Verified that the patch holds min containers.
> The min-held-containers constraint is not enforced during query runtime
> ------------------------------------------------------------------------
>
> Key: TEZ-2217
> URL: https://issues.apache.org/jira/browse/TEZ-2217
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.6.0, 0.7.0
> Reporter: Gopal V
> Assignee: Bikas Saha
> Attachments: TEZ-2217-debug.txt.bz2, TEZ-2217.1.patch, TEZ-2217.2.patch, TEZ-2217.3.patch, TEZ-2217.txt.bz2
>
>
> The min-held containers constraint is respected during query idle times, but is not respected when a query is actually in motion.
> The AM releases unused containers during dag execution without checking for min-held containers.
> {code}
> 2015-03-20 15:41:53,475 INFO [DelayedContainerManager] rm.YarnTaskSchedulerService: Container's idle timeout expired. Releasing container, containerId=container_1424502260528_1348_01_000013, containerExpiryTime=1426891313264, idleTimeoutMin=5000
> 2015-03-20 15:41:53,475 INFO [DelayedContainerManager] rm.YarnTaskSchedulerService: Releasing unused container: container_1424502260528_1348_01_000013
> {code}
> This is actually useful only after the AM has received a soft pre-emption message, doing it on an idle cluster slows down one of the most common query patterns in BI systems.
> {code}
> create temporary table smalltable as ...;
> select ... bigtable JOIN smalltable ON ...;
> {code}
> The smaller query in the beginning throws away the pre-warmed capacity.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)