[jira] [Created] (SPARK-32954) Let resource-managers yarn module can be tested independently

["Yang Jie (Jira)" <ji...@apache.org>]

Yang Jie created SPARK-32954:
--------------------------------

             Summary: Let resource-managers yarn module can be tested independently
                 Key: SPARK-32954
                 URL: https://issues.apache.org/jira/browse/SPARK-32954
             Project: Spark
          Issue Type: Improvement
          Components: YARN
    Affects Versions: 3.1.0
            Reporter: Yang Jie


When I tried to verify that the resource-managers yarn module passed all UTs in Scala 2.13 , I found that there are some problems blocking the test:
 # resource-managers/yarn module is not explicit dependency on sql/hive module but some case need HiveConf like `run Python application in yarn-client mode` in YarnClusterSuite, so we can't test resource-managers/yarn module independently as follow:
 ## mvn clean install -DskipTests -pl resource-managers/yarn -Pyarn -Pscala-2.13 -am

 ##  mvn clean test -pl resource-managers/yarn -Pyarn -Pscala-2.13

 # There are more than one `servlet-api` dependency in spark now, one is `javax.servlet:javax.servlet-api:3.10:compile` we config in core/pom.xml, the other is `jakarta.servlet:jakarta.servlet-api:4.0.3:test`  cascaded by `org.glassfish.jersey.test-framework.providers`, we can use `mvn dependency:tree` to check it .So when we execute resource-managers yarn module test independently, if the position of `javax.servlet:javax.servlet-api:3.10:compile` in the  in classpath is before `jakarta.servlet:jakarta.servlet-api:4.0.3:test`, there are some cases failed in YarnClusterSuite, YarnShuffleIntegrationSuite  and YarnShuffleAuthSuite



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org