You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "Jonathan Turner Eagles (Jira)" <ji...@apache.org> on 2021/04/23 21:27:00 UTC
[jira] [Commented] (TEZ-4192) Attach task specific info in
scheduler and retrieve in communicator
[ https://issues.apache.org/jira/browse/TEZ-4192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331061#comment-17331061 ]
Jonathan Turner Eagles commented on TEZ-4192:
---------------------------------------------
[~mustafaiman], this patch seems to be coming along. Are you looking for a review?
> Attach task specific info in scheduler and retrieve in communicator
> -------------------------------------------------------------------
>
> Key: TEZ-4192
> URL: https://issues.apache.org/jira/browse/TEZ-4192
> Project: Apache Tez
> Issue Type: Improvement
> Affects Versions: 0.9.2
> Reporter: Mustafa İman
> Assignee: Mustafa İman
> Priority: Major
> Attachments: TEZ-4192.1.patch, TEZ-4192.2.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> There is no way to pass information from scheduler to communicator. Sometimes we want to let communicator know about extra properties of the task other than its spec. One example is if a task is preemptable.
> When scheduler receives a task, its taskspec is already constructed. It is a nice immutable object at this point. We do not want to modify and put more info in it at that stage. So we pass a new piece of information to TaskSchedulerContext#taskAllocated along with the allocation. This info is passed to the communicator untouched. It is up to the communicator implementation to make use of it. Communicator implementation needs to override a new registerRunningTaskAttempt that accepts this additional object along with all the other arguments in the original registerRunningTaskAttempt method.
> The new registerRunningTaskAttempt method has a default implementation that falls back to the old method. So if the user does not override the new method, the same old behavior is preserved. Similarly on scheduler side, the user needs to call new taskAllocated method to pass the additional info object. If they use the old taskAllocated method, old behavior is preserved.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)