You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2023/06/10 08:19:00 UTC

[jira] [Commented] (YARN-8898) Fix FederationInterceptor#allocate to set application priority in allocateResponse

    [ https://issues.apache.org/jira/browse/YARN-8898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731177#comment-17731177 ] 

ASF GitHub Bot commented on YARN-8898:
--------------------------------------

slfan1989 opened a new pull request, #5732:
URL: https://github.com/apache/hadoop/pull/5732

   <!--
     Thanks for sending a pull request!
       1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
       2. Make sure your PR title starts with JIRA issue id, e.g., 'HADOOP-17799. Your PR title ...'.
   -->
   
   ### Description of PR
   JIRA: Fix FederationInterceptor#allocate to set application priority in allocateResponse
   
   In YARN-8898, we store the `ApplicationSubmissionContext` of a Job in the `FederationStateStore`, and this field can be `null`. However, in NodeManager's TestFederationInterceptor, the Application is submitted through a mock, which causes there to be no relationship between the `Application` and `HomeSubcluster` stored in the `MemoryFederationStateStore`. This does not affect the execution of the unit test, but the following information will be printed in the log.  
   
   ```
   2023-06-10 16:08:42,501 ERROR [pool-18-thread-1] utils.FederationStateStoreFacade (FederationStateStoreFacade.java:getApplicationSubmissionContext(1006)) - getApplicationSubmissionContext error, applicationId = application_123456_0001.
   org.apache.hadoop.yarn.server.federation.store.exception.FederationStateStoreException: Application application_123456_0001 does not exist.
   	at org.apache.hadoop.yarn.server.federation.store.utils.FederationStateStoreUtils.logAndThrowStoreException(FederationStateStoreUtils.java:172)
   	at org.apache.hadoop.yarn.server.federation.store.impl.MemoryFederationStateStore.getApplicationHomeSubCluster(MemoryFederationStateStore.java:294)
   	at org.apache.hadoop.yarn.server.federation.utils.FederationStateStoreFacade.getApplicationSubmissionContext(FederationStateStoreFacade.java:1001)
   	at org.apache.hadoop.yarn.server.nodemanager.amrmproxy.FederationInterceptor$2.run(FederationInterceptor.java:1283)
   	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   	at java.lang.Thread.run(Thread.java:748)
   ```
   
   We should avoid such information. In `TestFederationInterceptor#setup`, we initialize the relationship between `Application` and `HomeSubcluster` to avoid this issue.
   
   ### How was this patch tested?
   
   Junit Test.
   
   ### For code changes:
   
   - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
   - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
   - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files?
   
   




> Fix FederationInterceptor#allocate to set application priority in allocateResponse
> ----------------------------------------------------------------------------------
>
>                 Key: YARN-8898
>                 URL: https://issues.apache.org/jira/browse/YARN-8898
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Bibin Chundatt
>            Assignee: Shilun Fan
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 3.4.0
>
>         Attachments: YARN-8898.wip.patch
>
>
> In case of FederationInterceptor#mergeAllocateResponses skips application_priority in response returned



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org