[GitHub] [incubator-uniffle] smallzhongfeng commented on a diff in pull request #125: [Bugfix] When turn on AE, should not directly push it down to SortShu…

[GitBox <gi...@apache.org>]

smallzhongfeng commented on code in PR #125:
URL: https://github.com/apache/incubator-uniffle/pull/125#discussion_r937522454


##########
client-spark/spark2/src/main/java/org/apache/spark/shuffle/DelegationRssShuffleManager.java:
##########
@@ -72,6 +72,9 @@ private ShuffleManager createShuffleManagerInDriver() throws RssException {
         LOG.info("Use RssShuffleManager");
         return shuffleManager;
       } catch (Exception exception) {
+        if (sparkConf.getBoolean("spark.sql.adaptive.enabled", false)) {

Review Comment:
   ```
   public RssShuffleManager(SparkConf sparkConf, boolean isDriver) {
       if (sparkConf.getBoolean("spark.sql.adaptive.enabled", false)) {
         throw new IllegalArgumentException("Spark2 doesn't support AQE, spark.sql.adaptive.enabled should be false.");
       }
       this.sparkConf = sparkConf;
   ```
   When initializing `RssShuffleManager`, if AE is opened, an exception will be thrown here. `DelegationRssShuffleManager.createShuffleManagerInDriver()` catches the exception, prints the relevant log, and then pushes it down to`SortShuffleManager `. I think this is unreasonable.@jerqi
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org