You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Jose Luis López (Jira)" <ji...@apache.org> on 2026/10/03 08:23:00 UTC
[jira] [Updated] (YARN-12002) RouterClientRMService does not stop the delegation token secret manager on service stop
[ https://issues.apache.org/jira/browse/YARN-12002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jose Luis López updated YARN-12002:
-----------------------------------
Description:
#
##
### {\{RouterClientRMService#serviceStart}} calls \{{routerDTSecretManager.startThreads()}}, which starts the \{{ExpiredTokenRemover}} daemon thread of \{{AbstractDelegationTokenSecretManager}}. \{{serviceStop}} never calls \{{stopThreads()}}, so the thread outlives the service. It keeps calling the \{{FederationStateStoreFacade}} to roll master keys and remove expired tokens even after the Router and everything it owns have stopped.
The ResourceManager already handles this: \{{RMSecretManagerService#serviceStop}} stops its delegation token secret manager's threads. The Router leaves this step out.
There is a second problem in the same method: \{{serviceStop}} throws a \{{NullPointerException}} if the service was initialized but never started, because \{{userPipelineMap}} is only created in \{{serviceStart}}:
{code}
userPipelineMap.clear(); // NPE when serviceStart() never ran
{code}
*Proposed fix*
* In \{{serviceStop}}, after the RPC server stops, call \{{routerDTSecretManager.stopThreads()}} if the secret manager is not null.
* Guard \{{userPipelineMap.clear()}} against null.
*Tests*
Two new tests in \{{TestRouterClientRMService}}:
* \{{testServiceStopStopsDelegationTokenSecretManager}}: after \{{stop()}}, the secret manager reports \{{isRunning() == false}}.
* \{{testServiceStopWithoutStart}}: a service that was initialized but never started reaches \{{STOPPED}} without an exception.
Both tests fail without the fix and pass with it.
This was split out of YARN-11984. The federation mock Router there closes the state store after the Router stops, so it depends on this thread having stopped.
was:
### Description of PR
JIRA: [YARN-12002](https://issues.apache.org/jira/browse/YARN-12002)
`RouterClientRMService#serviceStart` calls `routerDTSecretManager.startThreads()`, which starts the `ExpiredTokenRemover` daemon thread of `AbstractDelegationTokenSecretManager`. `serviceStop` never called `stopThreads()`, so the thread outlived the service. It kept calling the `FederationStateStoreFacade` to roll master keys and remove expired tokens after the Router and everything it owns had stopped. `RMSecretManagerService#serviceStop` already does this for the ResourceManager; the Router left it out.
`serviceStop` also threw a `NullPointerException` when the service was initialized but never started, because `userPipelineMap` is only created in `serviceStart`.
Changes:
- `RouterClientRMService#serviceStop`: after the RPC server stops, call `routerDTSecretManager.stopThreads()` if the secret manager is not null.
- `RouterClientRMService#serviceStop`: guard `userPipelineMap.clear()` against null.
- `TestRouterClientRMService`: two new tests, `testServiceStopStopsDelegationTokenSecretManager` and `testServiceStopWithoutStart`.
This was split out of YARN-11984 ([#8681](https://github.com/apache/hadoop/pull/8681)), whose federation mock Router closes the state store after the Router stops and so depends on this thread having stopped. This PR is independent of the other PRs split out of YARN-11984 and can be reviewed and merged on its own.
### How was this patch tested?
Ran `TestRouterClientRMService` on Linux with JDK 21: 6/6 tests pass.
Reverted the fix in `RouterClientRMService` and kept the new tests. Both new tests fail:
- `testServiceStopStopsDelegationTokenSecretManager`: `expected: <false> but was: <true>`. The secret manager is still running after `stop()`.
- `testServiceStopWithoutStart`: `NullPointerException` on `userPipelineMap.clear()`.
### For code changes:
- [x] Does the title of this PR start with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: Have the integration tests been executed and the endpoint
declared according to the connector-specific documentation? *Note: Automated CI
testing doesn't cover all cases so manual testing with cloud storage is still
required.*
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files?
Object storage, new dependencies and LICENSE/NOTICE do not apply: this patch touches only the YARN Router and adds no dependencies.
### AI Tooling
If an AI tool was used:
- [x] The PR includes the phrase "Contains content generated by <tool>"
where <tool> is the name of the AI tool used.
- [x] My use of AI contributions follows the ASF legal policy
https://www.apache.org/legal/generative-tooling.html
Contains content generated by Claude Code.
> RouterClientRMService does not stop the delegation token secret manager on service stop
> ---------------------------------------------------------------------------------------
>
> Key: YARN-12002
> URL: https://issues.apache.org/jira/browse/YARN-12002
> Project: Hadoop YARN
> Issue Type: Bug
> Components: router
> Reporter: Jose Luis López
> Priority: Minor
>
> #
> ##
> ### {\{RouterClientRMService#serviceStart}} calls \{{routerDTSecretManager.startThreads()}}, which starts the \{{ExpiredTokenRemover}} daemon thread of \{{AbstractDelegationTokenSecretManager}}. \{{serviceStop}} never calls \{{stopThreads()}}, so the thread outlives the service. It keeps calling the \{{FederationStateStoreFacade}} to roll master keys and remove expired tokens even after the Router and everything it owns have stopped.
> The ResourceManager already handles this: \{{RMSecretManagerService#serviceStop}} stops its delegation token secret manager's threads. The Router leaves this step out.
> There is a second problem in the same method: \{{serviceStop}} throws a \{{NullPointerException}} if the service was initialized but never started, because \{{userPipelineMap}} is only created in \{{serviceStart}}:
> {code}
> userPipelineMap.clear(); // NPE when serviceStart() never ran
> {code}
> *Proposed fix*
> * In \{{serviceStop}}, after the RPC server stops, call \{{routerDTSecretManager.stopThreads()}} if the secret manager is not null.
> * Guard \{{userPipelineMap.clear()}} against null.
> *Tests*
> Two new tests in \{{TestRouterClientRMService}}:
> * \{{testServiceStopStopsDelegationTokenSecretManager}}: after \{{stop()}}, the secret manager reports \{{isRunning() == false}}.
> * \{{testServiceStopWithoutStart}}: a service that was initialized but never started reaches \{{STOPPED}} without an exception.
> Both tests fail without the fix and pass with it.
> This was split out of YARN-11984. The federation mock Router there closes the state store after the Router stops, so it depends on this thread having stopped.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org