You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2023/06/10 08:19:00 UTC
[jira] [Commented] (YARN-8898) Fix FederationInterceptor#allocate to set application priority in allocateResponse
[ https://issues.apache.org/jira/browse/YARN-8898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731177#comment-17731177 ]
ASF GitHub Bot commented on YARN-8898:
--------------------------------------
slfan1989 opened a new pull request, #5732:
URL: https://github.com/apache/hadoop/pull/5732
<!--
Thanks for sending a pull request!
1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
2. Make sure your PR title starts with JIRA issue id, e.g., 'HADOOP-17799. Your PR title ...'.
-->
### Description of PR
JIRA: Fix FederationInterceptor#allocate to set application priority in allocateResponse
In YARN-8898, we store the `ApplicationSubmissionContext` of a Job in the `FederationStateStore`, and this field can be `null`. However, in NodeManager's TestFederationInterceptor, the Application is submitted through a mock, which causes there to be no relationship between the `Application` and `HomeSubcluster` stored in the `MemoryFederationStateStore`. This does not affect the execution of the unit test, but the following information will be printed in the log.
```
2023-06-10 16:08:42,501 ERROR [pool-18-thread-1] utils.FederationStateStoreFacade (FederationStateStoreFacade.java:getApplicationSubmissionContext(1006)) - getApplicationSubmissionContext error, applicationId = application_123456_0001.
org.apache.hadoop.yarn.server.federation.store.exception.FederationStateStoreException: Application application_123456_0001 does not exist.
at org.apache.hadoop.yarn.server.federation.store.utils.FederationStateStoreUtils.logAndThrowStoreException(FederationStateStoreUtils.java:172)
at org.apache.hadoop.yarn.server.federation.store.impl.MemoryFederationStateStore.getApplicationHomeSubCluster(MemoryFederationStateStore.java:294)
at org.apache.hadoop.yarn.server.federation.utils.FederationStateStoreFacade.getApplicationSubmissionContext(FederationStateStoreFacade.java:1001)
at org.apache.hadoop.yarn.server.nodemanager.amrmproxy.FederationInterceptor$2.run(FederationInterceptor.java:1283)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
```
We should avoid such information. In `TestFederationInterceptor#setup`, we initialize the relationship between `Application` and `HomeSubcluster` to avoid this issue.
### How was this patch tested?
Junit Test.
### For code changes:
- [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files?
> Fix FederationInterceptor#allocate to set application priority in allocateResponse
> ----------------------------------------------------------------------------------
>
> Key: YARN-8898
> URL: https://issues.apache.org/jira/browse/YARN-8898
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Bibin Chundatt
> Assignee: Shilun Fan
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Attachments: YARN-8898.wip.patch
>
>
> In case of FederationInterceptor#mergeAllocateResponses skips application_priority in response returned
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org