[jira] [Commented] (TEZ-4650) Remove useless logic from AM plugin management: processSchedulerDescriptors

["László Bodor (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-4650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18029463#comment-18029463 ] 

László Bodor commented on TEZ-4650:
-----------------------------------

merged to master, thanks [~ayushtkn] for the review!

> Remove useless logic from AM plugin management: processSchedulerDescriptors
> ---------------------------------------------------------------------------
>
>                 Key: TEZ-4650
>                 URL: https://issues.apache.org/jira/browse/TEZ-4650
>             Project: Apache Tez
>          Issue Type: Improvement
>            Reporter: László Bodor
>            Assignee: László Bodor
>            Priority: Major
>             Fix For: 1.0.0
>
>          Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Proposal is to remove the method: [processSchedulerDescriptors|https://github.com/apache/tez/blob/0a5c7142c8ac5dff0fe6b18358bdb9eb99e2a678/tez-dag/src/main/java/org/apache/tez/dag/app/PluginManager.java#L198-L225]
> It is for two purposes:
> 1. If, according to a variable *isLocal*, local mode is enabled (so *uberEnabled*), checks if the task scheduler plugin descriptors list contain a corresponding *TezUber* entity. This is unnecessary, because if so, it's a bug in [parsePlugin|https://github.com/apache/tez/blob/0a5c7142c8ac5dff0fe6b18358bdb9eb99e2a678/tez-dag/src/main/java/org/apache/tez/dag/app/PluginManager.java#L170-L175] method, which ran previously and should have taken care about the same.
> 2. If, according to a variable, containers are enabled (so *tezYarnEnabled*), but there is no corresponding *TezYarn* entity in the descriptors, it magically adds it: so this fixes a bug where parsePlugin didn't take care of the same previously [here|https://github.com/apache/tez/blob/0a5c7142c8ac5dff0fe6b18358bdb9eb99e2a678/tez-dag/src/main/java/org/apache/tez/dag/app/PluginManager.java#L163-L168]
> This is just collection of hacks: in case an upstream application creates the ServicePluginsDescriptor properly (like hive does [here|https://github.com/apache/hive/blob/c338904b80d4dc6df08981738b69fa136bf6c624/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionState.java#L330-L350]), we should never hit this codepath. I'm assuming this is a leftover of early days of tez and local mode developments, like TEZ-2126: https://github.com/apache/tez/commit/7ef9dda732f124a234cbe8a665deb08c065f628f



--
This message was sent by Atlassian Jira
(v8.20.10#820010)