You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by GitBox <gi...@apache.org> on 2022/11/10 09:05:06 UTC

[GitHub] [accumulo] ctubbsii opened a new issue, #3075: Broken or Flaky test: AuthenticationTokenSecretManagerTest

ctubbsii opened a new issue, #3075:
URL: https://github.com/apache/accumulo/issues/3075

   **Test name(s)**
    - AuthenticationTokenSecretManagerTest.testVerifyPassword
   
   **Describe the failure observed**
   
   There seems to be some small probability that the AuthenticationTokenSecretManager will generate the same random password. I've seen the following error a few times, but it's hard to reproduce. Since it's calling into Hadoop code to generate things, this could depend on specific Hadoop versions.
   
   ```
   [ERROR] Tests run: 10, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 11.485 s <<< FAILURE! - in org.apache.accumulo.server.security.delegation.AuthenticationTokenSecretManagerTest
   [ERROR] org.apache.accumulo.server.security.delegation.AuthenticationTokenSecretManagerTest.testVerifyPassword  Time elapsed: 0.003 s  <<< FAILURE!
   org.opentest4j.AssertionFailedError: Different tokens for the same user shouldn't have the same password ==> expected: <false> but was: <true>
   	at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
   	at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
   	at org.junit.jupiter.api.AssertFalse.failNotFalse(AssertFalse.java:63)
   	at org.junit.jupiter.api.AssertFalse.assertFalse(AssertFalse.java:36)
   	at org.junit.jupiter.api.Assertions.assertFalse(Assertions.java:236)
   	at org.apache.accumulo.server.security.delegation.AuthenticationTokenSecretManagerTest.testVerifyPassword(AuthenticationTokenSecretManagerTest.java:222)
   
   ```
   
   **Additional context**
   
   If I remember correctly, this code is part of the delegation token management stuff that is used with Kerberos-enabled clusters so that MapReduce and other similar tasks don't need a keytab for each individual job task. Most recently, I saw this on 3.0.0-SNAPSHOT in the main branch, but I've previously seen it in development on earlier branches as well.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo] cshannon commented on issue #3075: Broken or Flaky test: AuthenticationTokenSecretManagerTest

Posted by GitBox <gi...@apache.org>.
cshannon commented on issue #3075:
URL: https://github.com/apache/accumulo/issues/3075#issuecomment-1332865490

   I took a look at this and ran the test in a loop in my IDE until it failed and it fails pretty fast after a few attempts. 
   
   The root cause is that the identifiers are the same between the two generated tokens. This is due to the fact that the issue date is part of the ID and is used for equality and it's set using System.currentTimeMillis() as shown below. 
   
   https://github.com/apache/accumulo/blob/b5701423330c8a55e50e8409d2492735149d70e0/server/base/src/main/java/org/apache/accumulo/server/security/delegation/AuthenticationTokenSecretManager.java#L81-L84
   
   This means that if the test runs fast enough and System.currentTimeMillis() returns the same time then the generated ID is the same and the test fails. A simple fix for the test itself is just sleeping 1 millisecond before generating the next token to guarantee a different timestamp and I'll do a quick PR for that.
   
   However, a follow on question/issue is whether or not this behavior is ok and if we want to update the Token id to be more random in some way so that this collision doesn't happen.
   
   I added some logging to illustrate the issue.
   
   #### Here are IDs from a successful run where the issueDate timestamps are different values:
   
   `ID1:TAuthenticationTokenIdentifier(principal:user@EXAMPLE.COM, keyId:1, issueDate:`**`1669849979940`**`, expirationDate:1669850039940, instanceId:6c7a4bc6-f7e6-45c2-be8a-4870d2e59fb5)`
   
   `ID2:TAuthenticationTokenIdentifier(principal:user@EXAMPLE.COM, keyId:1, issueDate:`**`1669849979950`**`, expirationDate:1669850039950, instanceId:6c7a4bc6-f7e6-45c2-be8a-4870d2e59fb5)`
   
   #### And here is a failed run where the issueDate timestamps are the same value:
   
   `ID1:TAuthenticationTokenIdentifier(principal:user@EXAMPLE.COM, keyId:1, issueDate:`**`1669849979983`**`, expirationDate:1669850039983, instanceId:e77af232-3d4b-4bf6-941a-5cfd7f3b25d5)`
   
   `ID2:TAuthenticationTokenIdentifier(principal:user@EXAMPLE.COM, keyId:1, issueDate:`**`1669849979983`**`, expirationDate:1669850039983, instanceId:e77af232-3d4b-4bf6-941a-5cfd7f3b25d5)`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo] EdColeman commented on issue #3075: Broken or Flaky test: AuthenticationTokenSecretManagerTest

Posted by GitBox <gi...@apache.org>.
EdColeman commented on issue #3075:
URL: https://github.com/apache/accumulo/issues/3075#issuecomment-1332884790

   Depending on the timer precision 1 millisecond may not be enough to guarantee the timestamp changes - it will depend on the underlying clock and currentTimeMillis() likely does not use nano seconds (that's why its faster)
   
   The sleep "may" work because of the additional overhead - but using a larger value (say > 65 millis) may force it to always be a new timestamp.  The 65 millis sticks in my head as being the worst case precision of a windows timer, but that may not be accurate anymore, or it was a different value.  But there is a precision difference between the reported milliseconds and the underlying clock so 1 millisecond may not guarantee a different value.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo] EdColeman commented on issue #3075: Broken or Flaky test: AuthenticationTokenSecretManagerTest

Posted by GitBox <gi...@apache.org>.
EdColeman commented on issue #3075:
URL: https://github.com/apache/accumulo/issues/3075#issuecomment-1332901133

   Yea - sorry. Using 100 millis is the tl/dr read version ;-)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo] cshannon commented on issue #3075: Broken or Flaky test: AuthenticationTokenSecretManagerTest

Posted by GitBox <gi...@apache.org>.
cshannon commented on issue #3075:
URL: https://github.com/apache/accumulo/issues/3075#issuecomment-1332898946

   Fair enough, I'll just bump it to 100 millis to play it safe, the test runs very fast so it isn't going to be a big deal.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo] cshannon closed issue #3075: Broken or Flaky test: AuthenticationTokenSecretManagerTest

Posted by GitBox <gi...@apache.org>.
cshannon closed issue #3075: Broken or Flaky test: AuthenticationTokenSecretManagerTest
URL: https://github.com/apache/accumulo/issues/3075


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org