You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2021/03/11 15:22:00 UTC
[jira] [Work logged] (HADOOP-16819) Possible inconsistent state of
AbstractDelegationTokenSecretManager
[ https://issues.apache.org/jira/browse/HADOOP-16819?focusedWorklogId=564668&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-564668 ]
ASF GitHub Bot logged work on HADOOP-16819:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 11/Mar/21 15:21
Start Date: 11/Mar/21 15:21
Worklog Time Spent: 10m
Work Description: steveloughran commented on a change in pull request #1894:
URL: https://github.com/apache/hadoop/pull/1894#discussion_r592451348
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenSecretManager.java
##########
@@ -356,16 +356,14 @@ private void updateCurrentKey() throws IOException {
int newCurrentId;
synchronized (this) {
newCurrentId = incrementCurrentKeyId();
- }
- DelegationKey newKey = new DelegationKey(newCurrentId, System
- .currentTimeMillis()
- + keyUpdateInterval + tokenMaxLifetime, generateSecret());
- //Log must be invoked outside the lock on 'this'
- logUpdateMasterKey(newKey);
- synchronized (this) {
- currentKey = newKey;
+ currentKey = new DelegationKey(newCurrentId, System
+ .currentTimeMillis()
+ + keyUpdateInterval + tokenMaxLifetime, generateSecret());
+
storeDelegationKey(currentKey);
}
+ //Log must be invoked outside the lock on 'this'
+ logUpdateMasterKey(currentKey);
Review comment:
so this is now happening after the store? And both generateSecret and store are now synchronized?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 564668)
Remaining Estimate: 0h
Time Spent: 10m
> Possible inconsistent state of AbstractDelegationTokenSecretManager
> -------------------------------------------------------------------
>
> Key: HADOOP-16819
> URL: https://issues.apache.org/jira/browse/HADOOP-16819
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, security
> Affects Versions: 3.3.0
> Reporter: Hankó Gergely
> Assignee: Hankó Gergely
> Priority: Major
> Attachments: HADOOP-16819.001.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> [AbstractDelegationTokenSecretManager.updateCurrentKey|https://github.com/apache/hadoop/blob/581072a8f04f7568d3560f105fd1988d3acc9e54/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenSecretManager.java#L360] increments the current key id and creates the new delegation key in two distinct synchronized blocks.
> This means that other threads can see the class in an *inconsistent state, where the key for the current key id doesn't exist (yet)*.
> For example the following method sometimes returns null when the token remover thread is between the two synchronized blocks:
> {noformat}
> @Override
> public DelegationKey getCurrentKey() {
> return getDelegationKey(getCurrentKeyId());
> }{noformat}
>
> Also it is possible that updateCurrentKey is called from multiple threads at the same time so *distinct keys can be generated with the same key id*.
>
> This issue is suspected to be the cause of the intermittent failure of [TestLlapSignerImpl.testSigning|https://github.com/apache/hive/blob/3c0705eaf5121c7b61f2dbe9db9545c3926f26f1/llap-server/src/test/org/apache/hadoop/hive/llap/security/TestLlapSignerImpl.java#L195] - HIVE-22621.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org