[spark] branch branch-3.3 updated: [SPARK-44581][YARN] Fix the bug that ShutdownHookManager gets wrong UGI from SecurityManager of ApplicationMaster

[ya...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

yao pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-3.3 by this push:
     new 330f7a789fb [SPARK-44581][YARN] Fix the bug that ShutdownHookManager gets wrong UGI from SecurityManager of ApplicationMaster
330f7a789fb is described below

commit 330f7a789fb9d3b1e501f180f477bf0ad578d10e
Author: 余良 <yu...@chinaunicom.cn>
AuthorDate: Wed Aug 9 13:46:57 2023 +0800

    [SPARK-44581][YARN] Fix the bug that ShutdownHookManager gets wrong UGI from SecurityManager of ApplicationMaster
    
    ### What changes were proposed in this pull request?
    
    I make the SecurityManager instance a lazy value
    
    ### Why are the changes needed?
    
    fix the bug in issue [SPARK-44581](https://issues.apache.org/jira/browse/SPARK-44581)
    
    **Bug:**
    In spark3.2 it throws the org.apache.hadoop.security.AccessControlException, but in spark2.4 this hook does not throw exception.
    
    I rebuild the hadoop-client-api.jar, and add some debug log before the hadoop shutdown hook is created, and rebuild the spark-yarn.jar to add some debug log when creating the spark shutdown hook manager, here is the screenshot of the log:
    ![image](https://github.com/apache/spark/assets/62563545/ea338db3-646c-432c-bf16-1f445adc2ad9)
    
    We can see from the screenshot, the ShutdownHookManager is initialized before the ApplicationManager create a new ugi.
    
    **Reason**
    
    The main cause is that ShutdownHook thread is created before we create the ugi in ApplicationMaster.
    
    When we set the config key "hadoop.security.credential.provider.path", the ApplicationMaster will try to get a filesystem when generating SSLOptions, and when initialize the filesystem during which it will generate a new thread whose ugi is inherited from the current process (yarn).
    After this, it will generate a new ugi (SPARK_USER) in ApplicationMaster and execute the doAs() function.
    
    Here is the chain of the call:
    ApplicationMaster.(ApplicationMaster.scala:83) -> org.apache.spark.SecurityManager.(SecurityManager.scala:98) -> org.apache.spark.SSLOptions$.parse(SSLOptions.scala:188) -> org.apache.hadoop.conf.Configuration.getPassword(Configuration.java:2353) -> org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:2434) -> org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:82)
    
    ### Does this PR introduce _any_ user-facing change?
    
    No
    
    ### How was this patch tested?
    
    I didn't add new UnitTest for this, but I rebuild the package, and runs a program in my cluster, and turns out that the user when I delete the staging file turns to be the same with the SPARK_USER.
    
    Closes #42405 from liangyu-1/SPARK-44581.
    
    Authored-by: 余良 <yu...@chinaunicom.cn>
    Signed-off-by: Kent Yao <ya...@apache.org>
    (cherry picked from commit e584ed4ad96a0f0573455511d7be0e9b2afbeb96)
    Signed-off-by: Kent Yao <ya...@apache.org>
---
 .../src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala b/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala
index 42e661cd47b..fcb38fe1def 100644
--- a/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala
+++ b/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala
@@ -80,7 +80,7 @@ private[spark] class ApplicationMaster(
 
   private val isClusterMode = args.userClass != null
 
-  private val securityMgr = new SecurityManager(sparkConf)
+  private lazy val securityMgr = new SecurityManager(sparkConf)
 
   private var metricsSystem: Option[MetricsSystem] = None
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org